1. Open Analytics NYC – 11/08/2012
Building Effective Frameworks
for Social Media Analysis

2. Agenda
• Social Media: An Intelligence perspective
• Common Analytic Pitfalls
• An Analytic Framework
• Case Study: Brand Management
– Problem Definition
– Source Selection
– Data Capture
– Data Reporting
– Data Analysis
• Ways Forward, Future Analysis
• Questions?

3. Intelligence
• Intelligence is information that has been
transformed to meet an operational need
Data Intelligence
Operational Lens

4. Intelligence Cycle
• No matter what methodology you use…
Collect
Distribute Store
Analyze
intelligence analysis is an iterative process.

5. Social Media: Intelligence Perspective
• Social Media Intelligence is a combination of the
best and worst features of:
– HUMINT
– OSINT
– SIGINT HUMINT
OSINT SIGINT

6. Social Media Analysis Goals
• Provide value to the organization – turn data into
intelligence using an “operational lens”
• Ensure cyclical feedback occurs during collection,
processing, analysis, and consumption
• Validate that a particular network is the right source
of data for the questions you need answered

7. Common Misconceptions
• Social media is not a panacea
– Not everyone uses social media
– Users of social media use it unevenly
– User behavior changes based on situations
• Just because people can talk about anything does
not mean they talk about everything all the time.

8. Common Pitfalls
• Analyzing What Instead of Why:
The important thing is often not what people are
saying… but why they are saying it.
• Using the Wrong Analysis Tools:
Reporting tools rarely help dig into the why. Many
common tools, reports, and metrics are actually
misleading:
– Word clouds atomize message context
– Sentiment metrics are often highly inaccurate
– Information in aggregate hides more than it reveals

9. Pitfalls: An Example of the Challenge

10. Pitfalls: An Example of the Challenge

11. Dangers of Disintegration
Source: Matthew Auer, Policy Studies Journal,
Volume 39, Issue 4, pages 709–736, Nov 2011

12. Analytic Framework
• Data Capture (DC) Capture
• Data Reporting (DR)
• Data Analysis (DA)
– What to measure Analyze Report
– What the data is saying
– What should be done based on the data
Source: Avinash Kaushik, Occam’s Razor Blog http://www.kaushik.net/avinash/web-analytics-consulting-
framework-smarter-decisions/

13. Choosing a Platform
• Social media, and the ways that it is used, is
relatively new and evolving rapidly:
– Static approaches to social media are flawed from
the outset
– No one metric or set of metrics will always let you
know what is happening
• Platforms need to be open and highly
adaptable to facilitate data capture, reporting,
and analysis

14. Case Study: Brand Management
• Industry: Gaming
– Experiencing 10% growth annually
– Overall revenue expected to exceed $80 billion by
2014
• In May, Zenimax Online Studios announced
Elder Scrolls Online
– Elder Scrolls V: Skyrim 2nd largest game of 2011

15. Problem Definition
• Question: How can brand managers use social
media to track and understand public
attitudes toward a product?
• Challenge: Capture relevant information for
social media sources.
– Query too large = false positives
– Query too small = miss potential information

16. Twitter
• Twitter has excellent analytical potential:
– Enormous volume, 400 million+ tweets per day
– Large user base, 140 million+ accounts
– Open API
• But its not without its limitations:
– 140 characters
– Limited historical (lookback) capacity without
using a 3rd party provider like DataSift or GNIP

17. Data Capture: Initial Query
• Twitter search for “Elder Scrolls Online”
– Simplest possible way to access information
– RSS feed for 10 days (Jun 27 – July 6 2012)

18. Data Capture: Entities & Associations
Hashtag TwitterHandle URL
Unstructured Keywords
Time / Date Stamp
Who What When Where
TwitterHandle Hashtags, Keywords, Time, Date Geo (if Available)
URLs

19. Data Reporting

20. Data Reporting

21. Data Analysis
• Analysis needs to be rooted in the operational
need:
“How can I use social media to track and
understand public attitudes toward my
product”
• Emphasis on hypothesis generation, testing,
and experimentation

22. Data Analysis: Hashtags
• Top hashtags were almost all generic or
abstract
– Undermines tracking and understanding
– Top hashtags tied to franchise, not to the game
Hashtags
#ElderScrolls #concept
#games #nerd
#online #geek
#MMO #gamer
#skyrim #ScreenShot

23. Data Analysis: Expanding the Query
• Hash tags from an initial subset of Tweets fed
back into the initial query
Initial Query Expanded Query
Results Results
Twitter Stream

24. Data Analysis: Sentiment
• Sentiment analysis on small snippets of text
like Tweets is generally poor
• Follow and convert linked URLs into derivative
sources
• Larger text sources offer potential value
with sentiment analysis
that tweets alone cannot
offer

25. Data Analysis: Sentiment
• Top negative and positive sentiment scores
can provide a glimpse into aggregate attitudes
• Provide starting points for additional analysis

26. Next Steps: Shape the Conversation
• Create and promote hashtags that help shape
the conversation and make it easier to collect
and analyze the Twitter stream

27. Next Steps: Segment the Data
• Segment, or cluster, your data by:
– User name or handle
– Hashtags
– Keywords
– Geographic region
to explore patterns and trends at the micro level
versus the entire dataset

28. Next Steps: Segment the Data

29. Next Steps: Graph Analysis

30. Lessons Learned
• Don’t:
– Try drinking from a fire hose, sometimes less
really is more;
– Use metrics you can’t tie to actions;
– Use visualizations or reports that strip the data
from its context.

31. Lessons Learned
• Do:
– Segment data rather than attempting to work in
the aggregate;
– Look for the why behind the message;
– Always return to the source material;
– Explore alternative explanations;
– Always consider the ultimate goal.

32. Thank You!
Craig Vitter
www.ikanow.com
cvitter@ikanow.com
github.com/ikanow/Infinit.e