Tag Maps

Slide 1: Ge ne rating Summarie s and Visualizatio n fo r Large Co lle ctio ns o f Ge o -re fe re nce d Pho to graphs Alexander Jaffe*, Mo r Naaman*, Tamir Tassa† , Marc Davis$ *Yahoo! Research Berkeley † Open University of Israel Yahoo! Research $

Slide 2: Attractio n Map o f Paris Stanle y Milgram , 1976. Psycho lo gical Maps o f Paris Generating Summaries - Mor Naaman 2

Slide 3: Attractio n Map o f Lo ndo n Jaffe e t al, 2006. Generating Summaries - Mor Naaman 3

Slide 4: Info rmatio n Ove rlo ad? Flickr “ge o tagge d” Generating Summaries - Mor Naaman 4

Slide 5: Ove rvie w • Problem definition • Intuition for solution • Algorithm for summarization • Visualizing the dataset • Evaluation • Demo? Generating Summaries - Mor Naaman 5

Slide 6: Pro ble m De finitio n Give n all pho to s fro m a ge o graphic re gio n, find a “re pre se ntative ” sum m ary se t • Dataset: (photo_id, user_id, latitude, longitude) (photo_id, tag) • Result: (photo_id, rank) Generating Summaries - Mor Naaman 6

Slide 7: Issue s to Tackle • Noisy data • Photographer biases – In locations – In Tags • Wrong data Whatever, color, city, spectrum, santa barbara, california, usa, Lookatme, Herbert Bayer Chromatic Gate Generating Summaries - Mor Naaman 7

Slide 8: Intuitio n Mo re “activity” in a ce rtain lo catio n indicate s im po rtance o f that lo catio n Tag that are unique to a ce rtain lo catio n can sugge st im po rtance o f that lo catio n Generating Summaries - Mor Naaman 8

Slide 9: (Ve ry) Simple Example Generating Summaries - Mor Naaman 9

Slide 10: Algo rithm Ove rvie w Hierarchical Clustering of the location • data Fo r e ach cluste generate cluster score For each cluster, r, ge ne rate cluste r • sco re Recursively generate ordering of all • Recursively generate ordering of photos in each cluster, based on all • photos in each cluster, based subcluster score and ordering on subcluster score and ordering Generating Summaries - Mor Naaman 10

Slide 11: The Cluste re d Re turn o f the (Ve ry) Simple Example ! 4,8,6,5,7 10 4, 6, 5 20 8,7 Generating Summaries - Mor Naaman 11

Slide 12: Ge ne rating a Summary • A complete ranking is produced for all photos in the dataset • An n-photo summary is simply the first n photos in this ranking. Generating Summaries - Mor Naaman 12

Slide 13: Ge ne rating Cluste r Sco re s • Main Factors: – Number of photos – Relevance (bias) factors – “Tag Distinguishability” “Tag Distinguishability” – “Photographer Distinguishability” Generating Summaries - Mor Naaman 13

Slide 14: Tag Distinguishability • A measure of uniqueness of concepts represented in the cluster (“document”) • TF/IDF based – Compute frequency of each tag (TF) – Compute (inverse) frequency of tag in the rest of the dataset (IDF) – Aggregate TF/IDF over all tags in cluster using L2 norm • Or, if you like formulas: Re ad the dam n pape r! Generating Summaries - Mor Naaman 14

Slide 15: Summary o f San Francisco Go lde n Gate Bridge TransAm e rica AT&T Base b all Park Go lde n Gate Tw in Pe aks Go lde n Gate O ce an Be ach Bay Bridge Chinato w n Generating Summaries - Mor Naaman 15

Slide 16: Pro gre ss Bar (almo st do ne ) • Problem definition • Intuition for solution • Algorithm for summarization • Visualizing the dataset • Evaluation • Demo? Generating Summaries - Mor Naaman 16

Slide 17: Tag Maps • Observation: – The algorithm identifies “representative” locations – The algorithm identifies unique, important tags Can be used to visualize the dataset! Generating Summaries - Mor Naaman 17

Slide 18: Tag Maps Generating Summaries - Mor Naaman 18

Slide 19: Tag Maps Generating Summaries - Mor Naaman 19

Slide 20: Ok, ho w do w e e valuate this? • Direct human-evaluation of algorithmic results – Evaluated Tag Maps with various weighting options – Compared summaries to 3 base conditions • Compared chosen locations to top 15 locations selected by humans (Milgram- style) Generating Summaries - Mor Naaman 20

Slide 21: Maybe w e have time fo r a de mo Generating Summaries - Mor Naaman 21

Slide 22: Maybe w e have time fo r Q’s http://zonetag.research.yahoo.com (applied in prototype cameraphone app) http://blog.yahooresearchberkeley.com (more on this and other topics) Become an intern, get involved: Email me. Mor Naaman mor@yahoo-inc.com Generating Summaries - Mor Naaman 22