SlideShare a Scribd company logo
1 of 23
Download to read offline
When UX Research Wasn’t
Enough
GRUBHUB RHIANNON ZIVIN NOV 2016
Background
A bit about
me.
2005 - 2007
2007 - 2010
2011 -
PRESENT
1999 - 2003
Majored in
Communication
Design
2003 - 2005
Worked in a
small design
studio in a large
corporation
Got a Masters
in Interaction
Design
Worked on a
variety of exciting
mobile products
Got my break
as a researcher
Meet
SOURCE: GRUBHUB.COM
Grubhub is the nation's leading online and mobile food ordering company
dedicated to connecting hungry diners with local takeout restaurants. The
company’s online and mobile ordering platforms allow diners to order directly
from more than 45,000 takeout restaurants in over 1,100 U.S. cities and
London. Every order is supported by the company’s 24/7 customer service
teams. Grubhub has offices in Chicago, New York and London.
Meet
SOURCE: GRUBHUB.COM
Grubhub is the nation's leading online and mobile food ordering company
dedicated to connecting hungry diners with local takeout restaurants. The
company’s online and mobile ordering platforms allow diners to order directly
from more than 45,000 takeout restaurants in over 1,100 U.S. cities and
London. Every order is supported by the company’s 24/7 customer service
teams. Grubhub has offices in Chicago, New York and London.
Meet
SOURCE: GRUBHUB.COM
Grubhub is the nation's leading online and mobile food ordering company
dedicated to connecting hungry diners with local takeout restaurants. The
company’s online and mobile ordering platforms allow diners to order directly
from more than 45,000 takeout restaurants in over 1,100 U.S. cities and
London. Every order is supported by the company’s 24/7 customer service
teams. Grubhub has offices in Chicago, New York and London.
2011 2012 2013 2014 2015 2016
90
Employees
1,300
Employees
IPO
GRUBMERGER
7.85
Million Active
Diners
271,100
Daily Average
Grubs
45,000
Restaurant
Partners
ACQ.
SERIES E
$50 MIL
SERIES D
$20 MIL
SERIES C
$11 MIL
2.58
Million Active
Diners
129,100
Daily Average
Grubs
17,000
Restaurant
Partners
Meet the rocketship
600
Employees
ACQ. ACQ.
2.58
Diners
129,100
Daily Orders
17,000
Restaurants
The Evolution of Research at Grubhub
2011 2012 2013 2014 2015 2016
90
Employees
1,300
EmployeesIPO
GRUBMERGER
UX
Research
Market
Research
7.85
Diners
271,100
Daily Orders
45,000
Restaurants
Evaluative
Research
Exploratory
Research
UX
Research
CX
Research
Grubhub
hires first
researcher
ACQ.
UX
Research
Market
Research
Grubhub
hires first
researcher
Part 1: UX Research Scales
2011 2012 2013 2014 2015 2016
90
Employees
1,300
EmployeesIPO
GRUBMERGER
7.85
Diners
271,100
Daily Orders
45,000
Restaurants
Evaluative
UX
Research
Exploratory
UX
Research
UX
Research
CX
Research
ACQ.
Challenges
1. How do we convince
devs and Product
Managers that research
is worthwhile?
2. If we empower non-
researchers to do
research won’t they
take our jobs?
3. Should market and
UX researchers work
together or draw lines
in the sand?
How do you convince Devs and Product
Managers that research is worthwhile?
● Exposing developers and product managers to the user in-person is
always the quickest way to win hearts and minds.
● Give them a predictable and consistent research process that they can
count on.
● Everyone can love research. Make it fun and exciting. Rally the troops,
celebrate findings and reward participation.
CHALLENGE 1
Rhiannon Zivin // When UX Research Wasn't Enough // Service Experience Conference 2016
Rhiannon Zivin // When UX Research Wasn't Enough // Service Experience Conference 2016
Rhiannon Zivin // When UX Research Wasn't Enough // Service Experience Conference 2016
Will empowering non-researchers with research
skills put you out of a job?
Heck no. That’s silly.
With proper training, non-researchers should be empowered do a variety of
research activities.
Out of bounds (examples)
Select the research method
Define research protocol
Select the size of the sample
Lead synthesis/analysis of the data
In bounds (examples)
Assist in the planning of a study
Help execute research (e.g.
moderate usability tests)
Participate in synthesis
✓
✓
✓
Ø
Ø
Ø
Ø
CHALLENGE 2
Should Market and UX Researchers work
together or draw lines in the sand?
Work together. The hard part is actually making it happen.
● Each group believes they own consumer data
● They prefer and find truth in different research methodologies
● Focus areas and philosophies are at odds
Macro, customer focused,
doing what’s best for the business
Micro, experience focused,
doing what’s best for the user
Understand human behavior, inform strategy
Market
Research
UX
Research
CHALLENGE 3
Evaluative
Research
Exploratory
Research
UX
Research
CX
Research
1,300
Employees
7.85
Diners
271,100
Daily Orders
45,000
Restaurants
2.58
Diners
129,100
Daily Orders
17,000
Restaurants
2011 2012 2013 2014 2015 2016
90
Employees IPO
GRUBMERGER
UX
Research
Market
Research
Grubhub
hires first UX
researcher
ACQ.
Part 2: UX Research Splits
Challenges
4. Do UX Researchers
belong in Marketing?
5. What’s the
difference between a
UX Researcher and a
(qual) Market
Researcher?
6. How did Grubhub
combine Market and
UX Research forces?
Perks
● Greater visibility into business processes and key metrics
● Better insight into brand positioning and how it relates to product
● Increased visibility with executives
Drawbacks
● Marketing thinks about people as “buyers”
● Desire for initiatives to tie back to ROI
● Don’t consider the entire customer journey,
such as customer service
Do UX Researchers belong in Marketing?
Customer?
CPA?
Attrition?
Positioning
?
LTV?
CHALLENGE 4
Rhiannon Zivin // When UX Research Wasn't Enough // Service Experience Conference 2016
UX Researchers can support marketing programs
that directly contribute to the bottomline.
Switching cups
What’s the difference between a UX and
Qualitative Market Researcher?
User Experience Researchers Qualitative Market Researchers
Behavioral and Attitudinal (What people do and why they do
it)
Attitudinal (What people say)
Focuses on user needs, goals, motivations, behaviors and
perceptions involved in the use of a product or service.
Focuses on customer wants, desires, aspirations,
perceptions, opinions, logic and trends involved in purchase
decisions.
Making products or services that satisfy a need, provide a
fluid experience and allow users to seamlessly achieve their
goals.
Formulating messages that resonate with the customer so
that the brand is top of mind when the use occasion arises.
Backgrounds in anthropology, design, psychology, HCI, etc. Backgrounds in marketing, business admin., psychology, etc.
Go beyond what people say they want and uncover
underlying needs. Problem solving.
Looking for holes and opportunities in the market to sell more
product.
CHALLENGE 5
How did Grubhub combine Market and UX
Research forces?
CHALLENGE 6
+
+
+
Market
Researchers
Customer
Experience
Quantitative
Research
User Experience
Researchers
Service
Design
Qualitative
Research
THANKS.
Rhiannon Zivin // When UX Research Wasn't Enough // Service Experience Conference 2016
Rhiannon Zivin // When UX Research Wasn't Enough // Service Experience Conference 2016

More Related Content

More from Adaptive Path

UI for Big Data Visualization | Jonathan Stray | UX Week 2012
UI for Big Data Visualization | Jonathan Stray | UX Week 2012UI for Big Data Visualization | Jonathan Stray | UX Week 2012
UI for Big Data Visualization | Jonathan Stray | UX Week 2012Adaptive Path
 
UX Week 2011: Only Good Touching
UX Week 2011: Only Good TouchingUX Week 2011: Only Good Touching
UX Week 2011: Only Good TouchingAdaptive Path
 
UX Week 2007: CNN.com Relaunch Case Study
UX Week 2007: CNN.com Relaunch Case StudyUX Week 2007: CNN.com Relaunch Case Study
UX Week 2007: CNN.com Relaunch Case StudyAdaptive Path
 
The importance of trust in customer experience
The importance of trust in customer experienceThe importance of trust in customer experience
The importance of trust in customer experienceAdaptive Path
 
Five Great Experiences to Nurture
Five Great Experiences to NurtureFive Great Experiences to Nurture
Five Great Experiences to NurtureAdaptive Path
 
Subject To Change: creating great products and services for an uncertain world
Subject To Change: creating great products and services for an uncertain worldSubject To Change: creating great products and services for an uncertain world
Subject To Change: creating great products and services for an uncertain worldAdaptive Path
 

More from Adaptive Path (8)

UI for Big Data Visualization | Jonathan Stray | UX Week 2012
UI for Big Data Visualization | Jonathan Stray | UX Week 2012UI for Big Data Visualization | Jonathan Stray | UX Week 2012
UI for Big Data Visualization | Jonathan Stray | UX Week 2012
 
UX Week 2011: Only Good Touching
UX Week 2011: Only Good TouchingUX Week 2011: Only Good Touching
UX Week 2011: Only Good Touching
 
UX Week 2007: CNN.com Relaunch Case Study
UX Week 2007: CNN.com Relaunch Case StudyUX Week 2007: CNN.com Relaunch Case Study
UX Week 2007: CNN.com Relaunch Case Study
 
The importance of trust in customer experience
The importance of trust in customer experienceThe importance of trust in customer experience
The importance of trust in customer experience
 
From the Outside-In
From the Outside-InFrom the Outside-In
From the Outside-In
 
Five Great Experiences to Nurture
Five Great Experiences to NurtureFive Great Experiences to Nurture
Five Great Experiences to Nurture
 
Subject To Change: creating great products and services for an uncertain world
Subject To Change: creating great products and services for an uncertain worldSubject To Change: creating great products and services for an uncertain world
Subject To Change: creating great products and services for an uncertain world
 
The Long Wow
The Long WowThe Long Wow
The Long Wow
 

Recently uploaded

Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsBlock Party
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024mikailaoh
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxb2kshani34
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comjakyjhon00
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazineRivanEleraki
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLkenzukiri
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfsaidbilgen
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...khushisharma298853
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before ConstructionResDraft
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTThink 360 Studio
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Ed Orozco
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfHctorFranciscoSnchez1
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxSamKuruvilla5
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxHasan S
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtTeeFusion
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Ted Drake
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillCre8iveskill
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Amil baba
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024Alan Dix
 

Recently uploaded (19)

Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teams
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptx
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.com
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazine
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before Construction
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPT
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdf
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptx
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy Shirt
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkill
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024
 

Rhiannon Zivin // When UX Research Wasn't Enough // Service Experience Conference 2016

  • 1. When UX Research Wasn’t Enough GRUBHUB RHIANNON ZIVIN NOV 2016
  • 2. Background A bit about me. 2005 - 2007 2007 - 2010 2011 - PRESENT 1999 - 2003 Majored in Communication Design 2003 - 2005 Worked in a small design studio in a large corporation Got a Masters in Interaction Design Worked on a variety of exciting mobile products Got my break as a researcher
  • 3. Meet SOURCE: GRUBHUB.COM Grubhub is the nation's leading online and mobile food ordering company dedicated to connecting hungry diners with local takeout restaurants. The company’s online and mobile ordering platforms allow diners to order directly from more than 45,000 takeout restaurants in over 1,100 U.S. cities and London. Every order is supported by the company’s 24/7 customer service teams. Grubhub has offices in Chicago, New York and London.
  • 4. Meet SOURCE: GRUBHUB.COM Grubhub is the nation's leading online and mobile food ordering company dedicated to connecting hungry diners with local takeout restaurants. The company’s online and mobile ordering platforms allow diners to order directly from more than 45,000 takeout restaurants in over 1,100 U.S. cities and London. Every order is supported by the company’s 24/7 customer service teams. Grubhub has offices in Chicago, New York and London.
  • 5. Meet SOURCE: GRUBHUB.COM Grubhub is the nation's leading online and mobile food ordering company dedicated to connecting hungry diners with local takeout restaurants. The company’s online and mobile ordering platforms allow diners to order directly from more than 45,000 takeout restaurants in over 1,100 U.S. cities and London. Every order is supported by the company’s 24/7 customer service teams. Grubhub has offices in Chicago, New York and London.
  • 6. 2011 2012 2013 2014 2015 2016 90 Employees 1,300 Employees IPO GRUBMERGER 7.85 Million Active Diners 271,100 Daily Average Grubs 45,000 Restaurant Partners ACQ. SERIES E $50 MIL SERIES D $20 MIL SERIES C $11 MIL 2.58 Million Active Diners 129,100 Daily Average Grubs 17,000 Restaurant Partners Meet the rocketship 600 Employees ACQ. ACQ.
  • 7. 2.58 Diners 129,100 Daily Orders 17,000 Restaurants The Evolution of Research at Grubhub 2011 2012 2013 2014 2015 2016 90 Employees 1,300 EmployeesIPO GRUBMERGER UX Research Market Research 7.85 Diners 271,100 Daily Orders 45,000 Restaurants Evaluative Research Exploratory Research UX Research CX Research Grubhub hires first researcher ACQ.
  • 8. UX Research Market Research Grubhub hires first researcher Part 1: UX Research Scales 2011 2012 2013 2014 2015 2016 90 Employees 1,300 EmployeesIPO GRUBMERGER 7.85 Diners 271,100 Daily Orders 45,000 Restaurants Evaluative UX Research Exploratory UX Research UX Research CX Research ACQ. Challenges 1. How do we convince devs and Product Managers that research is worthwhile? 2. If we empower non- researchers to do research won’t they take our jobs? 3. Should market and UX researchers work together or draw lines in the sand?
  • 9. How do you convince Devs and Product Managers that research is worthwhile? ● Exposing developers and product managers to the user in-person is always the quickest way to win hearts and minds. ● Give them a predictable and consistent research process that they can count on. ● Everyone can love research. Make it fun and exciting. Rally the troops, celebrate findings and reward participation. CHALLENGE 1
  • 13. Will empowering non-researchers with research skills put you out of a job? Heck no. That’s silly. With proper training, non-researchers should be empowered do a variety of research activities. Out of bounds (examples) Select the research method Define research protocol Select the size of the sample Lead synthesis/analysis of the data In bounds (examples) Assist in the planning of a study Help execute research (e.g. moderate usability tests) Participate in synthesis ✓ ✓ ✓ Ø Ø Ø Ø CHALLENGE 2
  • 14. Should Market and UX Researchers work together or draw lines in the sand? Work together. The hard part is actually making it happen. ● Each group believes they own consumer data ● They prefer and find truth in different research methodologies ● Focus areas and philosophies are at odds Macro, customer focused, doing what’s best for the business Micro, experience focused, doing what’s best for the user Understand human behavior, inform strategy Market Research UX Research CHALLENGE 3
  • 15. Evaluative Research Exploratory Research UX Research CX Research 1,300 Employees 7.85 Diners 271,100 Daily Orders 45,000 Restaurants 2.58 Diners 129,100 Daily Orders 17,000 Restaurants 2011 2012 2013 2014 2015 2016 90 Employees IPO GRUBMERGER UX Research Market Research Grubhub hires first UX researcher ACQ. Part 2: UX Research Splits Challenges 4. Do UX Researchers belong in Marketing? 5. What’s the difference between a UX Researcher and a (qual) Market Researcher? 6. How did Grubhub combine Market and UX Research forces?
  • 16. Perks ● Greater visibility into business processes and key metrics ● Better insight into brand positioning and how it relates to product ● Increased visibility with executives Drawbacks ● Marketing thinks about people as “buyers” ● Desire for initiatives to tie back to ROI ● Don’t consider the entire customer journey, such as customer service Do UX Researchers belong in Marketing? Customer? CPA? Attrition? Positioning ? LTV? CHALLENGE 4
  • 18. UX Researchers can support marketing programs that directly contribute to the bottomline. Switching cups
  • 19. What’s the difference between a UX and Qualitative Market Researcher? User Experience Researchers Qualitative Market Researchers Behavioral and Attitudinal (What people do and why they do it) Attitudinal (What people say) Focuses on user needs, goals, motivations, behaviors and perceptions involved in the use of a product or service. Focuses on customer wants, desires, aspirations, perceptions, opinions, logic and trends involved in purchase decisions. Making products or services that satisfy a need, provide a fluid experience and allow users to seamlessly achieve their goals. Formulating messages that resonate with the customer so that the brand is top of mind when the use occasion arises. Backgrounds in anthropology, design, psychology, HCI, etc. Backgrounds in marketing, business admin., psychology, etc. Go beyond what people say they want and uncover underlying needs. Problem solving. Looking for holes and opportunities in the market to sell more product. CHALLENGE 5
  • 20. How did Grubhub combine Market and UX Research forces? CHALLENGE 6 + + + Market Researchers Customer Experience Quantitative Research User Experience Researchers Service Design Qualitative Research

Editor's Notes

  1. Thanks for the wonderful introduction. It’s a pleasure to be here. As ____________ mentioned my name is Rhiannon (or Rhi for short) and I am a researcher at Grubhub. This talk, When UX Research Wasn’t Enough, tells the story of the research function at Grubhub, from the days of a small and scrappy startup to the public company that you know today.
  2. Before we dig into this story, let me give you some background on me, so you know where all of this is coming from. Background is in design. After graduating from CMU started my career in interaction design at Motorola Realized I was an OK designer but had a real passion for research. That’s where this story really begins...
  3. Show of hands, how many of you guys have heard of Grubhub? Ok, now how many of you have placed an order on Grubhub? For those of you who don’t know about it or haven’t used it, it’s simple. You enter your address into the system and Grubhub returns restaurants in your area that deliver to you. From my home in Lincoln Park there are 668 restaurants available From my office in the Loop there are 661 restaurants available From the Park 55 there are 437 restaurants available
  4. You select the restaurant, food you want to order, enter your cc information and place your order. Grubhub sends the order to the restaurant. The restaurant makes the food.
  5. ...and for about 20% of those orders are delivered by a Grubhub delivery partner. From a research perspective it’s a very fun place to be: We get to consider all sorts of interesting ordering situations: a couple staying in for the night, watching a movie, cuddling up and ordering some food to share a group of girls order some food to munch on as they get ready before going out for the night Or perhaps the conference goer, who wants to spend the evening in their hotel room, hair up in a sloppy bun, watching silly amounts of HGTV because her husband won’t let her have cable. Same goes for restaurants, we have over 45 thousand restaurant partners and there aren’t two restaurants that are alike. We have everything in our network from the hole in the wall to upper casual joints and we don’t just talk to the restaurant owners or managers, we talk to all of the employees from the hostess to the chef and everything in between watching them use our products. When it comes to our drivers, we’re in the back seat, watching them make deliveries...
  6. When I started on at Grubhub, little did I know how quickly Grubhub would grow over the next 6 years. I came on board in Q1 2011, during a major growth spurt. The founders had just raised a bunch of money in a series C round and then went on to raise even more money in a series D and E. By the end of 2011, we acquired DotMenu (included CampusFood) broadening our reach from 13 markets to hundred of markets across the country. In 2013 merged with our biggest competitor at the time, which eliminated threats of the competition and doubled the size of the company almost instantly. IPOed which allowed the company to raise a bunch of capital quickly to invest in things that will continue that growth. In 2015, started acquiring a slew of delivery companies that expanded our driver network allowing us to have more control over the delivery experience. These were some of the conditions that contributed to a growing research team.
  7. This diagram shows the progression of the research group over time, as the company grew. When I started on there were 90 some employees. I am the person in orange. They were just starting to build out their UX department. I was the first hire into that group. After me they hired a UX designer. A few of the projects I started on: we were just about to introduce pickup, so instead of getting something delivered you can pick it up instead. I tested a few of the new pickup flows. Another example: one of the devs created an Android app over the weekend and I tested it. The work was very tactical and focused on the product. When we merged with Seamless we inherited a market research function that reported into the then CEO of Seamless. For the most part UX and market research didn’t work together. In fact, when the annual diner survey came around each year and we were forced to work together we did so begrudgingly. In early 2015, research was separated from design, whereby design was absorbed into dev teams and research acted like a service to those teams. In 2015, with a new CMO onboard, the UX research team split into two teams: the exploratory side going into marketing and absorbed the responsibilities of the dwindling market research team, and the evaluative side staying in technology. The exploratory side, was forward thinking, looking for new ways to advance our products and the way people eat. The evaluative side was focused on evaluating our product concepts and designs. By the Summer of 2016, evaluative researchers felt stifled by their titles and wanted more, plus the org was starting to think of itself as more than just a product company, but now as a service. The timing was perfect. We were positioned to take on a new role in the organization: one we call CX. I’ll get into what this means for Grubhub later in the presentation.
  8. To get to each phase in our growth, we were faced with different challenges. Some of these challenges might sound very familiar, as I believe most research teams encounter them at some point or another. I’ll start with part one, a very young, UX research capability that was scaling with the growing demands of the company. How do we convince devs and product managers that research is worthwhile (twice, once when starting on, and again when merging with Seamless) Should we empower non-researchers to do research or will they take our jobs? When we merged with Seamless, we inherited a market research capability. How do we work together?
  9. Design is always down for research, it’s those pesky devs and product managers that are tough. Of the litany of things I’ve tried over time, these are the three things that made us the most successful at Grubhub. Standing alone, I don’t think any one of these things is novel. But in combination, these three things are consistent across research groups, making it a sort of special sauce. Quickly go through them.
  10. Here’s a very real example of how we make research fun at Grubhub. What you’re looking at here is Slice. Slice is Grubhub’s in-house restaurant that is worked, run and patroned by Grubhub employees. The purpose of Slice is to give employees first-hand exposure to what it’s like to use our restaurant-facing products in a realistic way. It’s one thing to use these products from the comfort of your desk, and it’s another thing altogether to use them while fulfilling orders, greeting dine-in guests, handing out pick-up orders and answering the phone. When viewed through this lens it’s almost a different product altogether. Slice allows us to give Grubhub employees exposure to restaurant products, helps them build empathy for the restaurant (hey it’s not as easy as I thought) test new features and allows employees to think about ways to improve them insitu. Slice runs every other Friday and teams of 6 work Slice. They start the shift with a short training on how we think about restaurants at Grubhub, they work Slice for an hour and then spend 30 minutes in a debrief talking about how things went and what could have gone better. People love it. Slice has been around since 2013.
  11. Researchers have come to me wondering in fear. If we empower them to do research, won’t we lose our jobs? Here’s what we’ve come across: Your time is stretched, yet a dev team wants to test a new design and there’s no time to wait. John Doe and Susie Q are doing some rogue research and you’re 99% sure they don’t know what they’re doing. Someone from another department is interested in a career change and wants to learn more about research. Absolutely. In fact, here are some stunning benefits: They’ll understand first-hand all of the hard work that goes into a study. You’ll make an ally in the org to help you tout the benefits of research. Trained non-researchers can help you to expand the breadth of your projects and expand your reach. Go into other markets, speak with more participants. They feel invested in the research and will want to socialize the learnings!
  12. This topic was front and center when we merged with Seamless. We had this new market research function, yet we had no idea how to work with them. They seemed to do things very differently than we did and we didn’t seem compatible. Some argue that Market and UX Research are in fact incompatible and that they should maintain this natural tension. However, I would argue that this tension can turn bad. These two teams are symbiotic. They need each other, in order to produce research that tells the whole story. To get around this, and to encourage more collaboration, today these teams are united under one team. I look for researchers who have an interest in the other discipline and are looking to grow in that area. Market Research: Aims to identify potential markets which includes understanding who will buy which product/services and opinions about a product/service. UX Research: User research focuses on user, his needs and environment around, how and in what circumstances users use products and services, and observes what people actually do. Help explain phenomena. Design Research is essentially more qualitative in nature - how do people use the product, what are their key needs etc. - which can further be converted into insights for conceptualizing better products ahead. Market Research on the other hand is more quantitative in nature - number of people buying one product vis-a-vis another one etc. - which might tell you the market performance of products, but won't answer the real 'why' behind them.
  13. Part 2: When the exploratory team, which was largely comprised of UX researchers moved into marketing, they experienced a major culture shock. Could they hang in this brand new world? As we started to expand the research team in New York we started to notice a new strain of UX Researcher, which were very different from the ones we were used to. Lastly, how are we comprised today. What does the CX function look like and in what ways is it a service design function?
  14. Sure, the culture shock sucks at times, but the change comes with clear benefits: The world opens and you see so much more into the business and how it works. I remember working with a designer at Grubhub who wondered why the CEO wasn’t worried about the feature he was working on. You have a better sense of how marketing thinks and works. You have greater visibility among executives When UX and marketing collaborate, beautiful things can happen.
  15. This is a picture of my son and I playing a game called Yummy Rummy. Yummy Rummy is Grubhub’s loyalty program that was developed at the end of 2011. I’d give you newer examples if it was allowed.
  16. One of the members of the board suggested that Grubhub develop a loyalty program. It wasn’t clear what type of program be that a points system or a punch card, but it had to be a loyalty program. So we pulled together the UX team + a creative from marketing and went to work, locking ourselves in a room for a week. We spent the first few days reading a bunch of secondary research, loyalty programs aren’t anything new why not learn from other’s mistakes? We had a small concession budget to work with and when we made the calculations, the amount one would receive per point or per punch was a pittance, they’d have to work very hard to earn the rewards, so we asked the question: if we can’t change the reward, can we change the perception of the reward. Enter the idea of variable rewards. The concept started with three bowls, the kind you’d see at… We actually tested the concept on employees. The old office was in a food desert, with only one restaurant nearby. People would go to it because it was easy and convenient despite the higher prices. Kind of like Grubhub. We tracked people’s purchases and setup a table near the elevator. Those with a receipt from the restaurant could play the game. We had three bowls and there was something under each bowl: there were jokes, fortunes and people had a 1 in 4 chance to win a gift card to the restaurant. People loved the game and would watch others play and it caused quite the buzz in the office. They would come up with these strategies. Oh, the one on the left is lucky. I always win with the one on the left. Some people liked the middle bowl. Others swore by the one on the right. We learned a lot from that study. Not only did we see a lift in the amount that people were going to the restaurant, but we also noticed areas to improve the game. For example, 1 in 4 chance to win was confusing and didn’t map to the three bowls we used in the game. For that reason we added a 4th option. But when we did that, people weren’t able to apply their strategies as easily. For that reason we gave each of the options, which turned into cards, it’s own identity. Each card hasn’t a different dessert on the front. My son always picks chocolate, not because it’s a winner, but because it’s his favorite.
  17. In fact, upon preliminary investigation we noticed that there was a lot of overlap between the two groups. They use a lot of the same methodologies: ethnographic studies, interviews, direct observation, usability testing, etc. In the same way that UX has become a loose term in the design community, we noted the same looseness on the research side. Market researcher use many of the same methodologies community is starting to adopt UX research methodologies such as Usability Testing. The differences on the surface may be difficult to discern. Entire ecosystem of the user. This includes the environment, actors and artifacts, needs and desires, barriers and blocks, core values, cultural aspects, and how all these interact and influence usage and purchase decisions. Deep, focused insight. Outside-in (UX), Inside-out (Market). Important component for product/service strategy (UX). Important component for business strategy (Market). Talks of the present, gain some insight from the past, avoid talks of the future (UX). Talks of the past, present and future are all fair game (Market). Do you want to investigate a current or potential product, service or brand positioning? Do your want to identify strengths and weaknesses in products? Understand purchasing decisions? Study reactions to advertising or marketing campaigns? Assess the usability of a website or other interactive services? Understand perceptions about the company, brand and product? Explore reactions to packaging and design?
  18. The CX Research team at Grubhub is an even mixture of quantitative and qualitative researchers. The quant side is made of market researchers, and the qual side is made of former UX researchers, ready to become service designers. This may not be the case for all organizations, but for those with leadership that is numbers minded, market research is familiar and comfortable, were as UX research acts as that fresh perspective that pushes the limits.