Tag Maps

Slideshow Transcript

1. Slide 1: Ge ne rating Summarie s and Visualizatio n fo r Large Co lle ctio ns o f Ge o -re fe re nce d Pho to graphs Alexander Jaffe*, Mo r Naaman*, Tamir Tassa† , Marc Davis$ *Yahoo! Research Berkeley † Open University of Israel Yahoo! Research $
2. Slide 2: Attractio n Map o f Paris Stanle y Milgram , 1976. Psycho lo gical Maps o f Paris Generating Summaries - Mor Naaman 2
3. Slide 3: Attractio n Map o f Lo ndo n Jaffe e t al, 2006. Generating Summaries - Mor Naaman 3
4. Slide 4: Info rmatio n Ove rlo ad? Flickr “ge o tagge d” Generating Summaries - Mor Naaman 4
5. Slide 5: Ove rvie w • Problem definition • Intuition for solution • Algorithm for summarization • Visualizing the dataset • Evaluation • Demo? Generating Summaries - Mor Naaman 5
6. Slide 6: Pro ble m De finitio n Give n all pho to s fro m a ge o graphic re gio n, find a “re pre se ntative ” sum m ary se t • Dataset: (photo_id, user_id, latitude, longitude) (photo_id, tag) • Result: (photo_id, rank) Generating Summaries - Mor Naaman 6
7. Slide 7: Issue s to Tackle • Noisy data • Photographer biases – In locations – In Tags • Wrong data Whatever, color, city, spectrum, santa barbara, california, usa, Lookatme, Herbert Bayer Chromatic Gate Generating Summaries - Mor Naaman 7
8. Slide 8: Intuitio n Mo re “activity” in a ce rtain lo catio n indicate s im po rtance o f that lo catio n Tag that are unique to a ce rtain lo catio n can sugge st im po rtance o f that lo catio n Generating Summaries - Mor Naaman 8
9. Slide 9: (Ve ry) Simple Example Generating Summaries - Mor Naaman 9
10. Slide 10: Algo rithm Ove rvie w Hierarchical Clustering of the location • data Fo r e ach cluste generate cluster score For each cluster, r, ge ne rate cluste r • sco re Recursively generate ordering of all • Recursively generate ordering of photos in each cluster, based on all • photos in each cluster, based subcluster score and ordering on subcluster score and ordering Generating Summaries - Mor Naaman 10
11. Slide 11: The Cluste re d Re turn o f the (Ve ry) Simple Example ! 4,8,6,5,7 10 4, 6, 5 20 8,7 Generating Summaries - Mor Naaman 11
12. Slide 12: Ge ne rating a Summary • A complete ranking is produced for all photos in the dataset • An n-photo summary is simply the first n photos in this ranking. Generating Summaries - Mor Naaman 12
13. Slide 13: Ge ne rating Cluste r Sco re s • Main Factors: – Number of photos – Relevance (bias) factors – “Tag Distinguishability” “Tag Distinguishability” – “Photographer Distinguishability” Generating Summaries - Mor Naaman 13
14. Slide 14: Tag Distinguishability • A measure of uniqueness of concepts represented in the cluster (“document”) • TF/IDF based – Compute frequency of each tag (TF) – Compute (inverse) frequency of tag in the rest of the dataset (IDF) – Aggregate TF/IDF over all tags in cluster using L2 norm • Or, if you like formulas: Re ad the dam n pape r! Generating Summaries - Mor Naaman 14
15. Slide 15: Summary o f San Francisco Go lde n Gate Bridge TransAm e rica AT&T Base b all Park Go lde n Gate Tw in Pe aks Go lde n Gate O ce an Be ach Bay Bridge Chinato w n Generating Summaries - Mor Naaman 15
16. Slide 16: Pro gre ss Bar (almo st do ne ) • Problem definition • Intuition for solution • Algorithm for summarization • Visualizing the dataset • Evaluation • Demo? Generating Summaries - Mor Naaman 16
17. Slide 17: Tag Maps • Observation: – The algorithm identifies “representative” locations – The algorithm identifies unique, important tags Can be used to visualize the dataset! Generating Summaries - Mor Naaman 17
18. Slide 18: Tag Maps Generating Summaries - Mor Naaman 18
19. Slide 19: Tag Maps Generating Summaries - Mor Naaman 19
20. Slide 20: Ok, ho w do w e e valuate this? • Direct human-evaluation of algorithmic results – Evaluated Tag Maps with various weighting options – Compared summaries to 3 base conditions • Compared chosen locations to top 15 locations selected by humans (Milgram- style) Generating Summaries - Mor Naaman 20
21. Slide 21: Maybe w e have time fo r a de mo Generating Summaries - Mor Naaman 21
22. Slide 22: Maybe w e have time fo r Q’s http://zonetag.research.yahoo.com (applied in prototype cameraphone app) http://blog.yahooresearchberkeley.com (more on this and other topics) Become an intern, get involved: Email me. Mor Naaman mor@yahoo-inc.com Generating Summaries - Mor Naaman 22