Marketers often have difficulty measuring the digital and experiential elements they build. Using three case studies, you’ll learn how to collect data from digital+live programs, how to analyze the data collected (i.e. vanity vs. pragmatic metrics) and how to use that information.
Speaker:
Ben Mcchesney, CTO, Helios Interactive Technologies
Slideshare has a character limit for descriptions so I've put the entire set of speaker notes on github :
https://gist.github.com/HeliosInteractive/7829375
---
Slide 2
My educational background is in design, development and economics.
My role at Helios is the strategic outlook and the establishment of our processes to become more effective and more efficient.
Slide 3
Our core competencies at Helios include retail , tradeshows , events, and permanent installations
We use a variety of technologies including touch , gesture, AR, and mobile platforms.
Slide 4
Where we differ from other small studios is that we do not create interactive art projects. Is it justifiable to create things that are just new and cool ? Why should clients work with us if we cannot measure how successful an event is ?
Like most of you in the audience we are working in many different environments. Where analytics start to get tricky is that we are not laser focused on single medium such as : digital signage, mobile applications, or microsites. So our metrics approach has to span across many different types of deployments.
Slide 5
We work with a wide variety of industries and a wide varieties of clients. Our methodologies apply across the board.
Slide 6
We are trying to achieve a better products.
Jeffery Liker explains Toyota's approach with continuous small improvements and small controllable batch sizes.
The Lean startup by Eric Reas applied the Toyota Method to Software Development.
Lean Analytics is an even more granular look at using data to build a better business.
Nate Silver is most known for his work as NY Times Elections Prognosticator and his work with statistics at ESPN.
Slide 7
BIG DATA , easy to get distracted from the data is telling.
We can measure almost anything measure but it’s important to measure what matters.
Slide 8
What are your goals ? And what does success looks like ?
this will involve a little bit of assumption and risk but without a yardstick for success numbers are less helpful.
Sound simple but is it ?
Slide 9
Quantitative data = numbers / spreadsheet.
Qualitative data = observational / feedback.
Slide 10
Vanity Metrics – CURRENT PROGRESS, big numbers, make you look good.
Actionable Metrics – guides future decisions. These are what will improve your experience.
If you are unsure – ask yourself : “How does this data change what we are doing?” If it doesn’t , or causes a panic : It’s a Vanity Metrics.
Slide 11
Create a memorable experience
remainder of notes available at https://gist.github.com/HeliosInteractive/7829375
9. Qualitative Data
Quantitative Data
Client Feedback
Number of users
Participant Feedback
Time stamped actions
Perception of Deployment
Participant Data
metric types | qualitative metrics vs. quantitative metrics
10. Vanity Metrics
Actionable Metrics
Total participants
% intent to share
Total facebook likes
Engagement time
Average users per day
Participant decision map
Total number of impressions
Shopping cart size
metric types | vanity metrics vs. actionable metrics
11. Emirates US Open | create a memorable experience
with the maximum amount of fans
Background in design, development and economics. ROLE : strategic outlook and the establishment of our processes to become more effective and more efficient.
Our core competencies retail , tradeshows , events, and permanent installationsWe use a variety of technologies including touch , gesture, AR, and mobile platforms.
Where we differ from other small studios : not interactive art projects. MOVE ON.Is it justifiable to create things that are just new and cool ? Why should clients work with us if we cannot measure how successful an event is ? Like probably most of you in the audience we are working in many different environments. Where analytics start to get tricky is that we are not laser focused on single medium such as : digital signage, mobile applications, or microsites. So our metrics solutions have to span across many different types of deployments.
We work with a wide variety of industries and a wide varieties of clients. This methodologies apply across the board.
There’s quite a few approaches for what we are trying to achieve : better products. INFLUENCED BY THESEJeffery Liker. MAIN PRINCIPLES OF : continuous small improvements and small controllable batch sizes.The Lean startup by Eric Reas APPLIED THESE TO STARTUPS AND SOFTWARE PRODUCT DEVELOPMENT.Lean Analytics is MORE GRANULAR. as Key Performance Indicators and specifically the data side of building a better business. Nate Silver is most known for his work as NY Times Elections Prognosticator and his work with statistics at ESPN. He’s one of the most recognized faces in statistics and his work merits looking at.Anyone interested in these sort of topics on data or analytics read these books their really interesting for their perspective.
BIG DATA , easy to get distracted from the data is telling.We can measure almost anything measure but it’s important to measure what matters.
What are your goals ? And what does success looks like ?this will involve a little bit of assumption and risk but without a yardstick for success numbers are less helpful.Sound simple but is it ?
Quantitative data = numbers / spreadsheet. Qualitative data = observational / feedback.
Vanity Metrics – CURRENT PROGRESS, big numbers, make you look good.Actionable Metrics – guides future decisions.These are what will improve your experience.If you are unsure – ask yourself : “How does this data change what we are doing?” If it doesn’t , or causes a panic : It’s a Vanity Metrics.
Create a memorable experience with the maximum amount of fans
We measured how many people created photos, how many had an intent to share, and how long on average it took people.
Contest Element and Prize Giveaways Experience took too long. Next steps - improve the overall length of the experience and have a higher throughput. Share station time is far to high, should be about 50% of experience time to prevent congestion
StubHub asked us to engage with their fans and create a multifaceted kiosk with a few different experiences.
DATA allowed pinpoint problem Improved internet, UX , more robust printer. Cutting the time in half because of the data we had.
Video wall with Stubhub Mascot and instagram NEAR side entrance. Annonymous face tracking used in digital signage.More development can let you know which content has the greatest reach.Are these numbers good ? We’re not quite sure. We’re establishing context to improve the experience in the future.
Activation was a success by focusing on the fans. Next steps – measure social media reach and the mascot interactions.
Very secretive, placed touch kiosks outside their booth.
Qualitative + Quantitative data.1 ) Engagement with the app was VERY LOW,2 ) Users stuck in the radial menu3 ) Product groups with the highest quality content had the best numbers.
TheEnterprise and tradeshow relationship is very different from fan and sports event one.We created a more inviting attract loop. The app was simply too busyTest and improve across a few events
There are many different ways to measure and it really depends on context. You should know going into your event.If your event is purely promotional for one week, it’s going to be really tough to gather enough data to see trends to improve it. The more iterations the more experimenting and using data can improve the event.Even if you are unsure how to use this data right now, it’s important to collect to use as contextual samples in the future.
