Ed H. Chi Google Research (Work done at Xerox PARC) CSCL2011 Keynote Abstract: Our research in Augmented Social Cognition is aimed at enhancing the ability of a group of people to remember, think, and reason. Our approach to creating this augmentation or enhancement is primarily model-driven. Our system developments are informed by models such as information scent, sensemaking, information theory, probabilistic models, and more recently, evolutionary dynamic models. These models have been used to understand a wide variety of user behaviors, from individuals interacting with social bookmark search in Delicious and MrTaggy.com to groups of people working on articles in Wikipedia. These models range in complexity from a simple set of assumptions to complex equations describing human and group behaviors. Indeed, increasingly, new social online resources such as social bookmarking sites and Wikis are becoming central in eLearning. By studying them, we further our understanding of how knowledge is constructed in a social context. In this talk, I will illustrate how a model-driven approach could help illuminate the path forward for social computing and social learning. -----

1. CSCL 2011 | Keynote
Augmented Social Cognition: How Social
Computing is Changing eLearning
Ed H. Chi
Google
Research
Work done while
at Palo Alto
Research Center
(PARC)
2008-05-13 CSCL 2011 Keynote

2. 2008-05-13 CSCL 2011 Keynote

3. Prelude:
A
personal
learning
story
To:
chi@acm.org
From:
Brad
Barrish
<brad@…removed.for.privacy….com>
Subject:
Pancreatic
cancer
Date:
Thu,
1
Feb
2007
21:37:55
PST
Hey
Ed.
I'm
a
fellow
del.icio.us
user
and
noticed
you
bookmark
a
lot
of
pancreatic
cancer
stuff.
I'm
at
home
with
my
dad
who
was
diagnosed
a
little
over
a
year
ago
and
is
now
at
the
tale
end
of
things.
I've
learned
a
lot
through
his
treatments
and
about
what's
out
there.
I
dunno
if
it's
something
you
or
a
family
member
has,
but
just
wanted
to
drop
you
an
email.
Be
well.
Brad
2008-05-13 CSCL 2011 Keynote

4. Talk
in
3
Acts
The
Importance
of
Social
Signals
in
eLearning
n Act
I:
The
Invisible
– Social
Search
n Act
II:
The
Visible
– Shared
Annotations
n Act
III:
The
Abstracted
– Shared
Knowledge
Space
2008-05-13 CSCL 2011 Keynote

5. Act
I:
Invisible
Social
Signals
from
the
Crowd
Joint
work
w/
Todd
Mytkowicz,
Rowan
Nairn,
Lawrence
Lee
[Chi
and
Mytkowicz,
Hypertext2008]
[Kammerer
et
al.,
CHI2009]
2008-05-13 CSCL 2011 Keynote

6. Using
Information
Theory
to
Model
Social
Tagging
[Ed
H.
Chi,
Todd
Mytkowicz,
ACM
Hypertext
2008]
Concepts
Topics
Users
Documents
Noise
Tags
Decoding
Encoding
T1…Tn
2008-05-13 CSCL 2011 Keynote

7. Tagging
Behavior
H(Tag)
shows
tag
saturation
H(Doc
|
Tag),
browsability
2008-05-13 CSCL 2011 Keynote

8. Implication
I(Doc;
Tag)
Mutual
Information
Raise
in
avg.
tag
/
bookmark
2008-05-13 CSCL 2011 Keynote

9. TagSearch:
MapReduce
Implementation
Tags URLs
P(URL|Tag)
P(Tag|URL)
n Spreading
Activation
in
a
bi-­‐graph
n Computation
over
a
very
large
data
set
– 150
Million+
bookmarks
2008-05-13 CSCL 2011 Keynote

10. TagSearch:
Use
Semantic
Analysis
to
Reduce
Noise
http://mrtaggy.com
Semantic Similarity Graph
Web
Tools
Reference
Guide
Howto
Tutorial
Tips
Help
Tip Tutorials
Tricks
2008-05-13 CSCL 2011 Keynote

11. 2008-05-13 CSCL 2011 Keynote

12. Experiment
Design
[Kammerer
et
al.
CHI2009]
n 2
interface
x
3
task
domain
design
– 2
Interface
(between-­‐subjects)
n Exploratory
vs.
Baseline
– 3
task
domains
(within-­‐subjects)
n Future
Architecture,
Global
Warming,
Web
Mashups
n 30
Subjects
(22
male,
8
female)
– Intermediate
or
advanced
computer
and
web
search
skills
– Half
assigned
Exploratory,
half
Baseline.
n For
each
domain,
single
block
with
3
task
types:
– Easy
and
Difficult
Page
Collection
Task
[6min
each]
– Summarization
Task
[12min]
– Keyword
Generation
Task
[2min]
2008-05-13 CSCL 2011 Keynote

13. Evauation
Results
[Kammerer
et
al.,
CHI2009]
n Exploratory
interface
users:
– performed
more
queries,
– took
more
time,
– wrote
better
summaries
(in
2/3
domains),
– generated
more
relevant
keywords
(in
2/3
domains),
and
– had
a
higher
cognitive
load.
n Suggestive
of
deeper
engagement
and
better
learning.
n Some
evidence
of
scaffolding
for
novices
in
the
keyword
generation
and
summarization
tasks.
2008-05-13 CSCL 2011 Keynote

14. Act
II:
Visible
Social
Signals
from
Shared
Highlighting
Kudos
to
Lichan
Hong,
Les
Nelson
[Hong
et
al,
AVI2008]
[Nelson
et
al.,
HCII
2009]
2008-05-13 CSCL 2011 Keynote

15. Finding
a
Restaurant
n Appropriate
for
the
occasion
2008-05-13 CSCL 2011 Keynote

16. Heuristics
Poor heuristic
Good heuristic
2008-05-13 CSCL 2011 Keynote

17. “Hints”
Solo
Cooperative (“good hints”)
2008-05-13 CSCL 2011 Keynote

18. SparTag.us:
Social
Highlighting
2008-05-13 CSCL 2011 Keynote

19. SparTag.us:
Social
Highlighting
n In
situ
tagging
while
reading
n Highlighting
n Shared
notebooking
n Sharing!
2008-05-13 CSCL 2011 Keynote

20. Highlighting
as
Importance
Indicator
recall first-visit
2008-05-13 CSCL 2011 Keynote

21. Evaluation
Task
&
Metric
[Nelson
et
al.,
HCII2009]
n Sensemaking
task
– Find
and
read
material
about
Enterprise
2.0
mashups
in
order
to
write
two
essays
n Seeds:
expert
content
for
scaffolding
– Tags
from
del.icio.us
– URLs
from
Google/PageRank
– Constructed
and
then
shared
through
social
mechanisms
(i.e.,
a
SparTag.us
friend )
n Performance
Measures
– Learning
gain:
Pre/Post
Knowledge
Test
Posttest score - Pretest score
Gain =
Max score - Pretest score
2008-05-13 CSCL 2011 Keynote

22. Procedure
SparTag.us
SF with
Friend
SparTag.us
Demographics
SO Only
Posttest
&
Pretest
Without
WS SparTag.us
2008-05-13 CSCL 2011 Keynote

23. Results:
Learning
Gain
N=18
SparTag.us
+
Friend
superior
to
both
individual
conditions
No
difference
between
the
two
control
conditions
2008-05-13 CSCL 2011 Keynote

24. URL
Kind
Code
Blog
B
Observation
Conference
C
Employment
E
My.Spartag.us
M
News
N
URL KindOpenSource
Code O
Blog Search
B S
Conference C
Vendor
V
Employment E
Wikipedia
MySpartagus M W
News Consultant
N X
OpenSource O
Search S
Vendor V
Wikipedia W
Consultant X
2008-05-13 CSCL 2011 Keynote

25. Von
Restorff
Isolation
Effect
[1933]
n As
applied
to
highlights,
the
von
Restorff
isolation
effect
suggests
that
readers:
n (a)
tend
to
focus
on
and
n (b)
learn
what
is
marked,
n whether
the
information
is
important
or
not.
– Nist
and
Hogrebe
87
2008-05-13 CSCL 2011 Keynote

26. Act
III:
Abstracted
Knowledge:
The
Science
of
Understanding
Wikipedia
Kudos
to
Bongwon
Suh,
Niki
Kittur
[Kittur
et
al.,
CHI2007]
[Suh
et
al.,
WikiSym
2009]
2008-05-13 CSCL 2011 Keynote

27. Exponential
Growth
of
Wikipedia:
an
accepted
‘fact’
Number of Articles (Log Scale)
http://en.wikipedia.org/wiki/Wikipedia:Modelling_Wikipedia’s_growth
2008-05-13 CSCL 2011 Keynote

28. Growth
of
Edits
2008-05-13 CSCL 2011 Keynote

29. Something
happened
in
early
2007
2008-05-13 CSCL 2011 Keynote

30. Growth
of
Active
Editors
*In thousands
2008-05-13 CSCL 2011 Keynote

31. Slowing
Growth
in
Global
Activity
*In thousands
2008-05-13 CSCL 2011 Keynote

32. Earlier
Exponential
Growth
Model
n Preferential
Attachment:
Edits
beget
edits
– more
number
of
previous
edits,
more
number
of
new
edits
Growth rate depends on:
N = current population
r = growth rate of the population
N(t) = N 0 " e rt
dN
= r" N
dt
Growth rate Current
of population !
population
!
2008-05-13 CSCL 2011 Keynote

33. Logistic
Growth
Model
n Ecological
population
growth
model
– Also
depend
on
environmental
conditions
– K,
carrying
capacity
(due
to
resource
limitation)
dN N
= rN(1" )
dt K
2008-05-13 CSCL 2011 Keynote

34. Match
to
Data:
#
of
New
Articles
n Follows
a
logistic
growth
curve
New Article
2008-05-13 CSCL 2011 Keynote

35. Struggle
for
Existence
-­‐
Darwin
n Biological
system
– Competition
increases
as
population
hit
the
limits
of
the
ecology
– Advantage
go
to
members
of
the
population
that
have
competitive
dominance
over
others
n Analogy
– Limited
opportunities
to
make
novel
contributions
– Increased
patterns
of
conflict
and
dominance
2008-05-13 CSCL 2011 Keynote

36. “Showering”
Hypothesis
What
drives
contributions
to
Wikipedia?
Cooperation
is
not
the
main
driver?
n Hypothesis:
Conflicts
drives
most
of
the
contributions.
– How
do
we
measure
conflicts?
n Conflicts
cause
coordination
costs
to
go
up.
– How
to
measure
coordination
costs?
n “negotiation
is
critical
to
helping
multiple
perspectives
to
converge
on
shared
knowledge.”
– Stahl,
Group
Cognition,
Ch8,
2004
2008-05-13 CSCL 2011 Keynote

37. Conflict/Coordination
Effects
in
Wikipedia
(Kittur, Suh, Pendleton, Chi, CHI2007)
2008-05-13 CSCL 2011 Keynote

38. Ratio
of
Reverted
Contributions
Monthly Ratio of Reverted Edits
2008-05-13 CSCL 2011 Keynote

39. Visual
Analytics
over
Wikipedia
data
Mediator
Pattern
-­‐
Terri
Schiavo
[Suh,
et
al.,
VAST2007]
Anonymous (vandals/
spammers)
Sympathetic to husband
Mediators
Sympathetic to parents
2008-05-13 CSCL 2011 Keynote

40. WikiDashboard.com
2008-05-13 CSCL 2011 Keynote

41. Coda:
A
Challenge:
A
modified
logistic
model
n Carrying
Capacity
as
a
function
of
time.
2008-05-13 CSCL 2011 Keynote

42. What
Did
We
Learn?
n The
Common
Thread:
– Utilization
of
Social
Signals
for
Learning
and
Information
Access
– Whether
it
is
invisible,
visible,
and
abstracted.
n The
Establishment
of
Common
Ground
– Implicit
Coordination
– Explicit
Coordination
– Negotiation
n “All
collective
actions
are
built
on
common
ground
and
its
accumulation.”
– Clark
and
Brennan,
1991
2008-05-13 CSCL 2011 Keynote

43. Research
Vision
Augmented
Social
Cognition
n Cognition:
the
ability
to
remember,
think,
and
reason;
the
faculty
of
knowing.
n Social
Cognition:
the
ability
of
a
group
to
remember,
think,
and
reason;
the
construction
of
knowledge
structures
by
a
group.
– (not
quite
the
same
as
in
the
branch
of
psychology
that
studies
the
cognitive
processes
involved
in
social
interaction,
though
included)
n Augmented
Social
Cognition:
Supported
by
systems,
the
enhancement
of
the
ability
of
a
group
to
remember,
think,
and
reason;
the
system-­‐supported
construction
of
knowledge
structures
by
a
group.
Citation:
Chi,
IEEE
Computer,
Sept
2008
2008-05-13 CSCL 2011 Keynote

44. From
Rote
Learning
to
Interaction
2008-05-13 CSCL 2011 Keynote

45. Thank
you!
n chi@acm.org
n http://edchi.net
2008-05-13 CSCL 2011 Keynote

46. What
I
will
not
talk
about
…
n Motivation
– Cultural
and
economic
incentives
– Personal
and
societal
values
– Psychology
(e.g.
cognitive,
personality,
social)
n Policy
and
Investment
– Resources
– Teacher
training
– Technological
investment
n With
the
Assumption
of
Motivation
and
Resources,
how
to
make
information
universally
accessible
and
useful
in
a
Web2.0
world?
2008-05-13 CSCL 2011 Keynote

47. 2008-05-13 CSCL 2011 Keynote

48. Lowering
Participation
/
Interaction
Costs
n Interaction
costs
# People willing to produce for “free”
determine
number
of
people
who
participate
n Surplus
of
attention
&
motivation
at
small
transaction
costs
n Therefore…
n Important
to
keep
interaction
costs
low
Cost of participation
2008-05-13 CSCL 2011 Keynote

49. Using
Machine
Learning
to
Detect
Conflicts
n Counting
‘Controversial’
labels
n 5x
cross-­‐validation,
R2
=
0.897
10000
9000
Actual controversial revisions
8000
7000
6000
5000
4000
3000
2000
1000
0
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
Predicted controversial revisions
2008-05-13 CSCL 2011 Keynote

50. Collaborative
Knowledge
Building
n “They
cannot
even
begin
to
coordinate
on
content
without
assuming
a
vast
amount
of
shared
information
or
common
ground….
And
to
coordinate
on
process,
they
need
to
update
their
common
ground
moment
by
moment.
All
collective
actions
are
built
on
common
ground
and
its
accumulation.”
– Clark
and
Brennan,
1991
n At
Web-­‐scale
social
learning,
what
we
know
about
the
nature
of
conflict
and
negotiation
is
woefully
inadequate.
2008-05-13 CSCL 2011 Keynote

51. Google
Plus
as
a
Research
Platform
2008-05-13 CSCL 2011 Keynote