Australian CIO Summit 2012: Big Data, New Physics, and Geospatial Super-Food by Tristan Sternson, Managing Director, InfoReady

Big Data,
New Physics, and
Geospatial Super-Food

Tristan Sternson, InfoReady Managing Director

1 © 2012 Infoready Pty Ltd

My Background – Tristan Sternson

Past 12 years focussed purely on IM / BI Solutions

Started InfoReady in 2008

Prior Roles – Accenture Data Management & Architecture / IM
Lead, PWC Consulting / IBM

Personally designed and deployed and led many large IM and
DW application in Australia and UK

Thought leader in Information Management in Australia and
APAC

Early adopter Data Governance, Big Data, Industry Data
Models, Appliance DW solutions


Who is InfoReady?

Pure-Play Information Management and Business Intelligence
Consulting firm
Team InfoReady career IM
and BI Experts
• One of the fastest growing
consulting firms in
Australia.
• IM Focused Tier One
Consulting capability.
• Focus - people, process and
technology
• Assisting companies turn
valuable information into
actionable intelligence.
• Strategy, Architecture,
Solution Design & Delivery

Big Data Definition

Datasets that grow so large that they become difficult to work with,
including; capture, storage, search, sharing, analytics, and visualization.

Benefits of working with larger and larger datasets allowing
analysts to "spot business trends, prevent diseases, combat crime.”

We haven’t seen anything yet, as more devices come online, eg;
mobile, airborn, logs, cameras, microphones etc…

4 Wikipedia - 2012
© 2012 Infoready Pty Ltd

The Big Data Opportunity

V3


Big Data – Why the hype?

By 2015, nearly 3B people will be online, pushing the data
created and shared to nearly 8 zettabytes.

30 billion pieces of content were added to Facebook this past
month by 600M plus users.

More than 2B videos were watched on YouTube … yesterday.

In the US mobile phone users between the ages of 18 and 24
send an incredible 110 text messages per day.

32B searches were performed last month … on Twitter.

Worldwide IP traffic will quadruple by 2015.


Business Value

Business leaders frequently make
1 in 3 decisions based on information they
don’t trust, or don’t have

1 in 2 access to leaders say theythey ’need to
Business
the information
don’t have

do their jobs

of CIOs cited “Business intelligence &
83% Analytics” as part of their visionary
plans to enhance competitiveness

of CEOs recognise they need to better
60% understand information more rapidly
in order to make swift decisions

7 7 © 2012 Infoready Pty Ltd

Big Data Trends

20% 80%

What the Industry Analysts say

Gartner predicts Big Data to be
one of the top-10 strategic initiatives
for 2012


What the Industry Analysts say

Key take-aways from Analyst perspectives Gartner TDWI
Data will grow exponentially

Fusion of structured and unstructured data

The connection between big data and advanced
analytics will get even stronger

Future users will not be able to put all useful
information into a single data warehouse


Enterprise Intelligence
vs. Enterprise Amnesia


Trend: Organizations Are Getting Dumber

Available
Observation
Computing Power Growth

Space

Context
Enterprise
Amnesia

Sensemaking
Algorithms

Time

Trend: Organizations Are Getting Dumber

Available
Observation

Space

WHY?
Context

Sensemaking
Algorithms

Time

Algorithms at Dead End.

You Can’t
Squeeze Knowledge
Out of a Pixel.


No Context

scrila34@msn.com


Context, definition

Better understanding
something by taking into
account the things around it.


Information in Context … and Accumulating

scrila34@msn.com

Job
Applicant Top 200
Customer

Criminal
Investigation
Identity
Thief


The Puzzle Metaphor

Imagine an ever-growing pile of puzzle pieces of varying sizes,
shapes and colors

What it represents is unknown – there is no picture on hand

Is it one puzzle, 15 puzzles, or 1,500 different puzzles?

Some pieces are duplicates, missing, incomplete, low quality, or
have been misinterpreted

Some pieces may even be professionally fabricated lies

Until you take the pieces to the table and attempt assembly,
you don’t know what you are dealing with


Puzzling

1000 pieces 100 pieces
100% 10%
(duplicates)

12 pieces 12 pieces
100% 100%
(pure noise)

66 pieces
66%


First Discovery – “we found Dora?”


Sorting Algorithm


Another Puzzle …


10 Mins – Completed Dora Puzzels


Data Finds Data


Obvious Duplicates in Front Of Your Eyes


Incremental Context – Incremental Discovery

10:00am START

1min “I can see Dora”

1min “How many puzzles are there?”

8min “Are there 1000 pieces and 3 or 4 puzzles?”

10min 2 x Dora puzzles complete

12min “I have blue sky and an animal”

18mins “The other puzzle is more colourful – maybe a red
motorbike”

23min “we’ve found Jenny Sanders – can I search google on my
iPhone for the picture?”

35min “How can we have 2 pieces the same?”

Lots of Sorted Pieces


Pieces in Context


Quickly we find meaning (90mins)

66 pieces
of
1190 pieces
only 5.5%


Wow 1%

11 pieces
of
1190 pieces
only 1%


Koala, Possum or Monkey?


Foundation


More Data Finds Data


Out of Tablespace…


Incremental Context – Incremental Discovery

55min “Second puzzle is definitely a motorbike – I can see a
wheel and seat”

65min Motorcycle coming together very quickly

70min “It’s definitely a koala”

75min “The koala has a baby”

83min “The middle piece of the bike is missing – do I really
need it, I know what it is”

88min “These are both Australian puzzles”

114min One of the kids starts isolating pieces that are causing her
“noise”

130min 7 chunks emerge from 7 piles of SORTED pieces

165min Pieces beginning to come together quite quickly and
picture starts to really emerge

How Context Accumulates

With each new observation … one of three assertions are made:
1) Un-associated; 2) placed near like neighbors; or 3) connected

Must favor the false negative

New observations sometimes reverse earlier assertions

Some observations produce novel discovery

As the working space expands, computational effort increases

Given sufficient observations, there can come a tipping point

Thereafter, confidence improves while computational effort
decreases!


Overstated Population
Unique Identities

True Population

Observations

Counting Is Difficult

Mark R Smith
(614) 13-123-123
DL: 00001234
Mark Smith
6/12/1978
0413123123

File 2

File 1


The Rise and Fall of a Population
Unique Identities

True Population

Observations

Data Triangulation

New Record
Mark R Smith
(614) 13-123-123
DL: 00001234
Mark Smith
6/12/1978
0413123123
Mark Randy Smith
0413123123
DL: 00001234

File 2

File 1


Big Data [in context]. New Physics.

More data: better the predictions
– Lower false positives
– Lower false negatives

More data: bad data good
– Suddenly glad your data is not perfect

More data: less compute


Big Data

Pile of ____ In Context

One Form of Context: “Expert Counting”

Is it 5 people each with 1 account … or is it 1
person with 5 accounts?

Is it 20 cases of H1N1 in 20 cities … or one
case reported 20 times?

If one cannot count … one cannot estimate
vector or velocity (direction and speed).

Without vector and velocity … prediction is
nearly impossible.

Expert Counting: Degrees of Difficulty
Deceit

Bob Jones Ken Wells
123455 550119

Incompatible
Features

Bob Jones bjones@hotmail
Fuzzy 123455

Bob Jones Robert T Jonnes
Exactly 123455 000123455
Same

Bob Jones Bob Jones
123455 123455

Key Features Enable Expert Counting

People Cars Router
Name Make Device ID
Address Model Make
Date of Birth Year Model
Phone License Plate No. Firmware Vers.
Passport VIN Asset ID
Nationality Owner Etc.
Biometric Etc.
Etc.


Consider Lying Identical Twins

PASSPORT #123
Sue PASSPORT #123
3/3/84 Sue
Uberstan 3/3/84
Exp 2011 Uberstan
Exp 2011

“Same
person –
trust me.”
Fingerprint

DNA
Most Trusted
Authority


The same thing cannot be in
two places … at the same
time.

Two different things cannot
occupy the same space … at
the same time.


Space Time Enables Absolute Disambiguation

People Cars Router
Name Make Device ID
When When When
Address Model Make
Where Where Where
Date of Birth Year Model
Phone License Plate No. Firmware Vers.
Passport VIN Asset ID
Nationality Owner Etc.
Biometric Etc.
Etc.


“Life Arcs” Are Also Telling

Bill Smith Bill Smith
13/4/67 13/4/67
Melbourne, Victoria Brisbane, Queensland

Address History Address History

Melbourne, Vic 2008-2008 Carina, QLD 2005-2009

St Kilda, Vic 2005-2008 Brisbane, QLD 2005-2005

Hampton, Vic 1996-2005 Bondi, NSW 1990-2005

Brighton, Vic 1984-1996 Carina, QLD 1982-1990


OMG


Space-Time-Travel

Cell phones are generating a staggering amount of
geo-locational data – 600B transactions per day
being created in the US alone

This data is being “de-identified” and shared with
third parties – in volume and in real-time

Your movement quickly reveals where you spend
your time (e.g., evenings vs. working hours)

Re-identification (figuring out who is who) is
somewhat trivial


Powerful Predictions

Prediction with 87% certainty where you will be
next Thursday at 5:35pm

Names of the top 10 people you co-locate with,
not at home and not at work

Intelligence service preempts the next mass
protest in real-time

Robbery of a convenience store is about to
happen at 10:42pm


Consequences

Space-time-travel data is the ultimate
biometric

It will enable enormous opportunity

It will unravel one’s secrets

It will challenge existing notions of privacy

And, it’s here now and more to come


Macro Trends


The Greater the Context, the Greater the Value

Data
in Context
Value of Data

Pile of Data

(Big) Records Managed (Ludicrous Big)

Time Is Of The Essence
The better the
predictions … the
Batch faster they will be
wanted.
Day
Willingness to Wait

“Why did we have
to wait until the
Hour end of the day for
the smart answer?”

200ms Real-Time

(Iffy) Relevance (Totally)

Enterprise Intelligence
One Plausible Journey
One Plausible Journey


Sense and Respond

Observation
Space

New
Observations

What you know


Sense and Respond

Observation
Space

Data Finds
Data

Relevance
Finds the Sensor
(200ms)
?
Decide


Sense and Respond Explore and Reflect

Observation
Space Deep
Reflection

Curated
Data

Data Finds Pattern
Data Discovery

Directed
Attention
Relevance
Finds the Sensor
(200ms)
?
Relevance
Decide Find You



Observation
Space Deep
Reflection

Curated
Data

Data Finds Pattern
Data Discovery

Directed
Attention
Relevance NEW
Finds the Sensor
(200ms)
? INTERESTS

Decide



Observation
Space Deep
Reflection

Curated
Data

Data Finds Pattern
Data Discovery

Directed
Attention
Relevance NEW
Finds the Sensor
(200ms)
? INTERESTS

Decide

64
Report and Manage
© 2012 Infoready Pty Ltd

Closing Thoughts


The most competitive organizations

are going to make sense of what they are observing

fast enough to do something about it

while they are observing it.


Wish This On The Enemy

Available
Observation

Space

Context
Enterprise
Amnesia

Sensemaking
Algorithms

Time

The Way Forward: Enterprise Intelligence

Available
Observation

Space
Context

Sensemaking
Algorithms

Time

Questions?

Email: tristan.sternson@infoready.com.au
Twitter: http://www.twitter.com/tsternson
Blog: www.infoready.com.au
LinkedIn: http://www.linkedin.com/in/tristansternson


Australian CIO Summit 2012: Big Data, New Physics, and Geospatial Super-Food by Tristan Sternson, Managing Director, InfoReady

Recommended

Recommended

More Related Content

What's hot

What's hot (14)

Viewers also liked

Viewers also liked (15)

Similar to Australian CIO Summit 2012: Big Data, New Physics, and Geospatial Super-Food by Tristan Sternson, Managing Director, InfoReady

Similar to Australian CIO Summit 2012: Big Data, New Physics, and Geospatial Super-Food by Tristan Sternson, Managing Director, InfoReady (20)

More from IT Network marcus evans

More from IT Network marcus evans (20)

Recently uploaded

Recently uploaded (20)

Australian CIO Summit 2012: Big Data, New Physics, and Geospatial Super-Food by Tristan Sternson, Managing Director, InfoReady