Querying and managing complex networks

1. Luiz Gomes-Jr
querying & managing complex networks

2. Luiz Gomes-Jr
Thesis Defense
Supervisor: André Santanchè
querying and managing complex networks
Institute of Computing
Unicamp
October 2015

3. Overview
• Motivation/Objective
• Complex Data Management System
• Query language
• Processing model
• Optimization
• Management
• Applications
• Conclusion 3

4. Connected world
4

5. 5
Arvind Dorai Lily White...
Andy Curtis
Bryanne M
Grant Simm...
Danny Gues...
Mohsen Sha...
Laura Martí
Patrick Kl...
Hossein Fa...
Remi Bierm... Byron Marks
Nabeel Far...
Leon De Kl...
Michael Ar...
Chris Kalman
Melanie Mc...
Olivier Ng...
Pablo Pilo...
Jon Rigg
Peyman Afs...
Rebecca Jo...
Daniel Gui...
Waleed Hwy
Umar Faroo...
Evandro Ba...
Eduardo El...
Stuart Mel...
Arthur Cas...
Nathan Chow
Paulo Henr...
Amin Kamal...
Serhan Reb...
Diogenes V...
Jan Mateu ...
Sanderson ...
Robin Chri...
Joanne Kruk Eric Sethna
Marcus Bar...
Jurandy So...
Daiana Silva
Wesley Mes...
Mehdi Vafaei
Glauber Ca...
Alessandro...
Leonardo S...
Ahmed Atau...
Flavio Cunto
Saiedeh Sa...
Andre Naime
Rodrigo Ia...
Andrew Kane Babak Ebra...
Camilo Jacob
Derya Demi...
Juliana Ba...
Juliana Ga...
Paulo Lojk...
Guilherme ...
Johnnatan ...
Yann James
Irene Pivo...
Natalie Rios
Rodrigo Ma...
Jeff Pound
Walmir Amo...
Javier Mon...
Alan Taitz
Ansis Rosm...
Aaditeshwa...
Tim Schnei...
Eduin Muñoz
Kayo Yoshida
Bill Fraga
Yelda Turkan
Crístian D...
Ivan Rodol...
Marcio Mat...
Julio Césa...
Anna-Lina ...
Rafael Lot...
Leandro Za...
Fabien Ben...
Marcilio M...
Cesar G. C...
Tony Huynh
Fernando M...
Keivan Ahm...
Peter Nelson
Cristina M...
Steve McFa...
Daniel De ...
Roohen S. ...
Leonardo R...
Sabrina de...
Shahin Kam...
Pia Brunetti
Alex Breda...
Heike Isem...
Diandra Pi...
Vitor E. S...
Samad Baza...
Maria Lore...
Mohammad R...
Renato Sil...
Sonia Waha...
Dorothy Fu...
Nielsen Ca...
Juliana Za...
Tim Benke
Bernardo V...
Shahab Kam...
Sara Peyroux
Xavier Pérez
Vahid Fallah
Carlos Edu...
Saman Mh
Pourya Sha...
Fernanda A...
Christophe...
Nima Rezaei
Nádia Pk
Ricardo Da...
Michal Skiba
Tyler Evans
Fabio Dias
Amanda Pe ...
Marcel Stein
Jiewen Wu
Diego De F...
João Ricardo
Florencia ...
Iury Porto
Andréa Pit...
Lucas Ferr...
Juliana Em...
Angel Pont...
Diorgenes ...
Holger Linke
Elio Brune...
Alexander ...
Paulo Casado
Elisangela...
Erica Band...
Fernanda T...
Gilbia Por...
Evelyn Chris
Elisabete ...
Evandro Du...
Danilo Lac...
Ricardo Pa...
Andre Costa
Javier Req...
Thiago Tho...
Xerxes Zam...
Laura Mart...
Gabriela Ü...
Roberta Graf
Ivo Koga
Alessandra...
Andreas Br...
Fabiola Bu...
Daniele Co...
Paula Krüger
Bruno Silv...
Veronica M...
Fernando S...
Felicia Bo...
Suzana Mar...
Bruno Lenh...
Mariella B...
Rodrigo Di...
Benjamin L...
Carolina B...
Júlia Farr...
Fabricia B...
José Augus...
Juliana Fr...
Júllia Ted...
Vítor Bacc...
Rafael Lopes
Rodrigo Vi...
Erika Mend...
Roberto Bi...
Gabriel Sa...
Gustavo Ja...
Renato Tra...
Luiz Cláudio
Claudia St...
Mariana Pe...
Kristina H...
Moises Alm...
Dânia Naomi
Lukas Posn...
Luca Diniz...
Rafael Art...
Daniel Linux
Danilo Mor...Thiago Ton...
Wilson Tal...
Vanessa St...
Luiz Ferna...
Caro Recal...
Monika Gro...
Marilia Br...
Yudai Naka...
Laura Man...
Denise Tel...
Roney Celi...
Luana Rebo...
Helen Siman
Cleber Val...
Cristiano ...
Jociel And...
Renato Fil...
Bruno Vilar
Ivana Borges
Refugio Co...
João Sávio...
Taísa Santos
Afsaneh Na...
Wilian Flá...
Nelson Rel...
Lidia Gald...
Marcio Jul...
Eric Haine...
Lucas C Fr...
Facundo Pe...
Andre Fiorot
Viviane Al...
André Alme...
Fernanda B...
Adriana Le...
Nelio Pinh...
Arthur Gás...
Eduardo Za...
Edson Burgo
Joana Mala...
Susanne Fr...
Evie Negro
Felipe Car...
Hugo Rafacho
Josmenio F...
Alex Menegão
Saray Chu
Humberto S...
Rodrigo Sa...
Artur Grover
Isabella P...
Edu Pedro
Naoki Arima
Leonardo M...
Raquel Siman
Ivelize Ro...
Vitor Lécio
Marcelo El...
Victor Dam...
Caio Salom...
Mari Maroc...
Lívia Aizawa
Diogo Rebit
Xuu Zhang
Daniel Lap...
Rafael Seco Natassia F...
Arthur Cyp...
Edi Agrellos
Ieda Manfre
Gilberto P...
Fabrício A...
Peruzzo Fo...
Du Sandrini
Ciana Quei...
Silvana Li...
Lucas Mello
Alexandre ...
Halysson A...
Sandro Souza
Ariel Vargas
Ana Siman
Hugo de So...
Nes-k Flow...
Redi Sique...
Sheila Mor...
Douglas Es...
Eduardo Ca...
Bruno Abreu
Renata Rav...
Joyce Luca...
Telma Santos
Marcio Gomes
Danilo San...
Oswaldo Ba...
Tiago Santos
Scheila Fa...
Robledo Se...
Julia Siman
Cynthia Ra...
Pedro Mosc...
Ingrid Nunes
Renato Cés...
Vitor A. F...
Leandro Fr...
Thiago Kar...
Tiago Bita...
Isabella S...
Joy Gazzon...
Gabriel Ca...
Fábio Fabre
Ewerton Si...
Neeylson M...
Vanessa Du...
Cláudio Ca...
Edson Borin
Roberta Si...
Randerson ...
Hatun Mach...
Ricardo To...
Josilene S...
Sérgio Fre...
Ulisses Fu...
Luciola Go...
Joatane So...
Vivian Alv...
Matheus Mota
Rafael Rez...
Jociel Per...
Thiago Godoi
Ralph Dard...
Elisa Costa
Rafael Piu...
Ádria Bentes
Luciano An...
Hilário Se...
Dijalma Fa...
Patricia C...
Josmercio ...
Luiza Mart...
Mogai Ti
Marcelo Pr...
Hudson Zanin
Ace Es
Tiago Amar...
Gabriel Va...
Emilio Bed...
Oscar Esga...
Tatiane Pi...
Jeane Alme...
Dulcinea P...
André Sant...
Fabiano Ruy
Victor Mar...
Daniel Rocco
Thiago Ras...
Felipe Sertã
Daniel Cug...
Renato Bes...
Paulo Bern...
Felipe Alves
Carlos Ale...
Laryssa Fl...
Ronaldo Lo...
Cristian S...
Fecatep ES
Mateuzin F...
Nina Fraga
Lukinha Ro...
Jefersson ...
Braian Car...
Rita Juliato
Bruno Card...
Miguel Di ...
Frederico ...
Vetinha Fr...
Helder Dias
Neumar Mal...
Fabiano Si...
Josmenio F...
Pedra Viva
Jeidi Siman
Ecological Networks

6. 6
Arvind Dorai Lily White...
Andy Curtis
Bryanne M
Grant Simm...
Danny Gues...
Mohsen Sha...
Laura Martí
Patrick Kl...
Hossein Fa...
Remi Bierm... Byron Marks
Nabeel Far...
Leon De Kl...
Michael Ar...
Chris Kalman
Melanie Mc...
Olivier Ng...
Pablo Pilo...
Jon Rigg
Peyman Afs...
Rebecca Jo...
Daniel Gui...
Waleed Hwy
Umar Faroo...
Evandro Ba...
Eduardo El...
Stuart Mel...
Arthur Cas...
Nathan Chow
Paulo Henr...
Amin Kamal...
Serhan Reb...
Diogenes V...
Jan Mateu ...
Sanderson ...
Robin Chri...
Joanne Kruk Eric Sethna
Marcus Bar...
Jurandy So...
Daiana Silva
Wesley Mes...
Mehdi Vafaei
Glauber Ca...
Alessandro...
Leonardo S...
Ahmed Atau...
Flavio Cunto
Saiedeh Sa...
Andre Naime
Rodrigo Ia...
Andrew Kane Babak Ebra...
Camilo Jacob
Derya Demi...
Juliana Ba...
Juliana Ga...
Paulo Lojk...
Guilherme ...
Johnnatan ...
Yann James
Irene Pivo...
Natalie Rios
Rodrigo Ma...
Jeff Pound
Walmir Amo...
Javier Mon...
Alan Taitz
Ansis Rosm...
Aaditeshwa...
Tim Schnei...
Eduin Muñoz
Kayo Yoshida
Bill Fraga
Yelda Turkan
Crístian D...
Ivan Rodol...
Marcio Mat...
Julio Césa...
Anna-Lina ...
Rafael Lot...
Leandro Za...
Fabien Ben...
Marcilio M...
Cesar G. C...
Tony Huynh
Fernando M...
Keivan Ahm...
Peter Nelson
Cristina M...
Steve McFa...
Daniel De ...
Roohen S. ...
Leonardo R...
Sabrina de...
Shahin Kam...
Pia Brunetti
Alex Breda...
Heike Isem...
Diandra Pi...
Vitor E. S...
Samad Baza...
Maria Lore...
Mohammad R...
Renato Sil...
Sonia Waha...
Dorothy Fu...
Nielsen Ca...
Juliana Za...
Tim Benke
Bernardo V...
Shahab Kam...
Sara Peyroux
Xavier Pérez
Vahid Fallah
Carlos Edu...
Saman Mh
Pourya Sha...
Fernanda A...
Christophe...
Nima Rezaei
Nádia Pk
Ricardo Da...
Michal Skiba
Tyler Evans
Fabio Dias
Amanda Pe ...
Marcel Stein
Jiewen Wu
Diego De F...
João Ricardo
Florencia ...
Iury Porto
Andréa Pit...
Lucas Ferr...
Juliana Em...
Angel Pont...
Diorgenes ...
Holger Linke
Elio Brune...
Alexander ...
Paulo Casado
Elisangela...
Erica Band...
Fernanda T...
Gilbia Por...
Evelyn Chris
Elisabete ...
Evandro Du...
Danilo Lac...
Ricardo Pa...
Andre Costa
Javier Req...
Thiago Tho...
Xerxes Zam...
Laura Mart...
Gabriela Ü...
Roberta Graf
Ivo Koga
Alessandra...
Andreas Br...
Fabiola Bu...
Daniele Co...
Paula Krüger
Bruno Silv...
Veronica M...
Fernando S...
Felicia Bo...
Suzana Mar...
Bruno Lenh...
Mariella B...
Rodrigo Di...
Benjamin L...
Carolina B...
Júlia Farr...
Fabricia B...
José Augus...
Juliana Fr...
Júllia Ted...
Vítor Bacc...
Rafael Lopes
Rodrigo Vi...
Erika Mend...
Roberto Bi...
Gabriel Sa...
Gustavo Ja...
Renato Tra...
Luiz Cláudio
Claudia St...
Mariana Pe...
Kristina H...
Moises Alm...
Dânia Naomi
Lukas Posn...
Luca Diniz...
Rafael Art...
Daniel Linux
Danilo Mor...Thiago Ton...
Wilson Tal...
Vanessa St...
Luiz Ferna...
Caro Recal...
Monika Gro...
Marilia Br...
Yudai Naka...
Laura Man...
Denise Tel...
Roney Celi...
Luana Rebo...
Helen Siman
Cleber Val...
Cristiano ...
Jociel And...
Renato Fil...
Bruno Vilar
Ivana Borges
Refugio Co...
João Sávio...
Taísa Santos
Afsaneh Na...
Wilian Flá...
Nelson Rel...
Lidia Gald...
Marcio Jul...
Eric Haine...
Lucas C Fr...
Facundo Pe...
Andre Fiorot
Viviane Al...
André Alme...
Fernanda B...
Adriana Le...
Nelio Pinh...
Arthur Gás...
Eduardo Za...
Edson Burgo
Joana Mala...
Susanne Fr...
Evie Negro
Felipe Car...
Hugo Rafacho
Josmenio F...
Alex Menegão
Saray Chu
Humberto S...
Rodrigo Sa...
Artur Grover
Isabella P...
Edu Pedro
Naoki Arima
Leonardo M...
Raquel Siman
Ivelize Ro...
Vitor Lécio
Marcelo El...
Victor Dam...
Caio Salom...
Mari Maroc...
Lívia Aizawa
Diogo Rebit
Xuu Zhang
Daniel Lap...
Rafael Seco Natassia F...
Arthur Cyp...
Edi Agrellos
Ieda Manfre
Gilberto P...
Fabrício A...
Peruzzo Fo...
Du Sandrini
Ciana Quei...
Silvana Li...
Lucas Mello
Alexandre ...
Halysson A...
Sandro Souza
Ariel Vargas
Ana Siman
Hugo de So...
Nes-k Flow...
Redi Sique...
Sheila Mor...
Douglas Es...
Eduardo Ca...
Bruno Abreu
Renata Rav...
Joyce Luca...
Telma Santos
Marcio Gomes
Danilo San...
Oswaldo Ba...
Tiago Santos
Scheila Fa...
Robledo Se...
Julia Siman
Cynthia Ra...
Pedro Mosc...
Ingrid Nunes
Renato Cés...
Vitor A. F...
Leandro Fr...
Thiago Kar...
Tiago Bita...
Isabella S...
Joy Gazzon...
Gabriel Ca...
Fábio Fabre
Ewerton Si...
Neeylson M...
Vanessa Du...
Cláudio Ca...
Edson Borin
Roberta Si...
Randerson ...
Hatun Mach...
Ricardo To...
Josilene S...
Sérgio Fre...
Ulisses Fu...
Luciola Go...
Joatane So...
Vivian Alv...
Matheus Mota
Rafael Rez...
Jociel Per...
Thiago Godoi
Ralph Dard...
Elisa Costa
Rafael Piu...
Ádria Bentes
Luciano An...
Hilário Se...
Dijalma Fa...
Patricia C...
Josmercio ...
Luiza Mart...
Mogai Ti
Marcelo Pr...
Hudson Zanin
Ace Es
Tiago Amar...
Gabriel Va...
Emilio Bed...
Oscar Esga...
Tatiane Pi...
Jeane Alme...
Dulcinea P...
André Sant...
Fabiano Ruy
Victor Mar...
Daniel Rocco
Thiago Ras...
Felipe Sertã
Daniel Cug...
Renato Bes...
Paulo Bern...
Felipe Alves
Carlos Ale...
Laryssa Fl...
Ronaldo Lo...
Cristian S...
Fecatep ES
Mateuzin F...
Nina Fraga
Lukinha Ro...
Jefersson ...
Braian Car...
Rita Juliato
Bruno Card...
Miguel Di ...
Frederico ...
Vetinha Fr...
Helder Dias
Neumar Mal...
Fabiano Si...
Josmenio F...
Pedra Viva
Jeidi Siman
Social Networks

7. 7
Arvind Dorai Lily White...
Andy Curtis
Bryanne M
Grant Simm...
Danny Gues...
Mohsen Sha...
Laura Martí
Patrick Kl...
Hossein Fa...
Remi Bierm... Byron Marks
Nabeel Far...
Leon De Kl...
Michael Ar...
Chris Kalman
Melanie Mc...
Olivier Ng...
Pablo Pilo...
Jon Rigg
Peyman Afs...
Rebecca Jo...
Daniel Gui...
Waleed Hwy
Umar Faroo...
Evandro Ba...
Eduardo El...
Stuart Mel...
Arthur Cas...
Nathan Chow
Paulo Henr...
Amin Kamal...
Serhan Reb...
Diogenes V...
Jan Mateu ...
Sanderson ...
Robin Chri...
Joanne Kruk Eric Sethna
Marcus Bar...
Jurandy So...
Daiana Silva
Wesley Mes...
Mehdi Vafaei
Glauber Ca...
Alessandro...
Leonardo S...
Ahmed Atau...
Flavio Cunto
Saiedeh Sa...
Andre Naime
Rodrigo Ia...
Andrew Kane Babak Ebra...
Camilo Jacob
Derya Demi...
Juliana Ba...
Juliana Ga...
Paulo Lojk...
Guilherme ...
Johnnatan ...
Yann James
Irene Pivo...
Natalie Rios
Rodrigo Ma...
Jeff Pound
Walmir Amo...
Javier Mon...
Alan Taitz
Ansis Rosm...
Aaditeshwa...
Tim Schnei...
Eduin Muñoz
Kayo Yoshida
Bill Fraga
Yelda Turkan
Crístian D...
Ivan Rodol...
Marcio Mat...
Julio Césa...
Anna-Lina ...
Rafael Lot...
Leandro Za...
Fabien Ben...
Marcilio M...
Cesar G. C...
Tony Huynh
Fernando M...
Keivan Ahm...
Peter Nelson
Cristina M...
Steve McFa...
Daniel De ...
Roohen S. ...
Leonardo R...
Sabrina de...
Shahin Kam...
Pia Brunetti
Alex Breda...
Heike Isem...
Diandra Pi...
Vitor E. S...
Samad Baza...
Maria Lore...
Mohammad R...
Renato Sil...
Sonia Waha...
Dorothy Fu...
Nielsen Ca...
Juliana Za...
Tim Benke
Bernardo V...
Shahab Kam...
Sara Peyroux
Xavier Pérez
Vahid Fallah
Carlos Edu...
Saman Mh
Pourya Sha...
Fernanda A...
Christophe...
Nima Rezaei
Nádia Pk
Ricardo Da...
Michal Skiba
Tyler Evans
Fabio Dias
Amanda Pe ...
Marcel Stein
Jiewen Wu
Diego De F...
João Ricardo
Florencia ...
Iury Porto
Andréa Pit...
Lucas Ferr...
Juliana Em...
Angel Pont...
Diorgenes ...
Holger Linke
Elio Brune...
Alexander ...
Paulo Casado
Elisangela...
Erica Band...
Fernanda T...
Gilbia Por...
Evelyn Chris
Elisabete ...
Evandro Du...
Danilo Lac...
Ricardo Pa...
Andre Costa
Javier Req...
Thiago Tho...
Xerxes Zam...
Laura Mart...
Gabriela Ü...
Roberta Graf
Ivo Koga
Alessandra...
Andreas Br...
Fabiola Bu...
Daniele Co...
Paula Krüger
Bruno Silv...
Veronica M...
Fernando S...
Felicia Bo...
Suzana Mar...
Bruno Lenh...
Mariella B...
Rodrigo Di...
Benjamin L...
Carolina B...
Júlia Farr...
Fabricia B...
José Augus...
Juliana Fr...
Júllia Ted...
Vítor Bacc...
Rafael Lopes
Rodrigo Vi...
Erika Mend...
Roberto Bi...
Gabriel Sa...
Gustavo Ja...
Renato Tra...
Luiz Cláudio
Claudia St...
Mariana Pe...
Kristina H...
Moises Alm...
Dânia Naomi
Lukas Posn...
Luca Diniz...
Rafael Art...
Daniel Linux
Danilo Mor...Thiago Ton...
Wilson Tal...
Vanessa St...
Luiz Ferna...
Caro Recal...
Monika Gro...
Marilia Br...
Yudai Naka...
Laura Man...
Denise Tel...
Roney Celi...
Luana Rebo...
Helen Siman
Cleber Val...
Cristiano ...
Jociel And...
Renato Fil...
Bruno Vilar
Ivana Borges
Refugio Co...
João Sávio...
Taísa Santos
Afsaneh Na...
Wilian Flá...
Nelson Rel...
Lidia Gald...
Marcio Jul...
Eric Haine...
Lucas C Fr...
Facundo Pe...
Andre Fiorot
Viviane Al...
André Alme...
Fernanda B...
Adriana Le...
Nelio Pinh...
Arthur Gás...
Eduardo Za...
Edson Burgo
Joana Mala...
Susanne Fr...
Evie Negro
Felipe Car...
Hugo Rafacho
Josmenio F...
Alex Menegão
Saray Chu
Humberto S...
Rodrigo Sa...
Artur Grover
Isabella P...
Edu Pedro
Naoki Arima
Leonardo M...
Raquel Siman
Ivelize Ro...
Vitor Lécio
Marcelo El...
Victor Dam...
Caio Salom...
Mari Maroc...
Lívia Aizawa
Diogo Rebit
Xuu Zhang
Daniel Lap...
Rafael Seco Natassia F...
Arthur Cyp...
Edi Agrellos
Ieda Manfre
Gilberto P...
Fabrício A...
Peruzzo Fo...
Du Sandrini
Ciana Quei...
Silvana Li...
Lucas Mello
Alexandre ...
Halysson A...
Sandro Souza
Ariel Vargas
Ana Siman
Hugo de So...
Nes-k Flow...
Redi Sique...
Sheila Mor...
Douglas Es...
Eduardo Ca...
Bruno Abreu
Renata Rav...
Joyce Luca...
Telma Santos
Marcio Gomes
Danilo San...
Oswaldo Ba...
Tiago Santos
Scheila Fa...
Robledo Se...
Julia Siman
Cynthia Ra...
Pedro Mosc...
Ingrid Nunes
Renato Cés...
Vitor A. F...
Leandro Fr...
Thiago Kar...
Tiago Bita...
Isabella S...
Joy Gazzon...
Gabriel Ca...
Fábio Fabre
Ewerton Si...
Neeylson M...
Vanessa Du...
Cláudio Ca...
Edson Borin
Roberta Si...
Randerson ...
Hatun Mach...
Ricardo To...
Josilene S...
Sérgio Fre...
Ulisses Fu...
Luciola Go...
Joatane So...
Vivian Alv...
Matheus Mota
Rafael Rez...
Jociel Per...
Thiago Godoi
Ralph Dard...
Elisa Costa
Rafael Piu...
Ádria Bentes
Luciano An...
Hilário Se...
Dijalma Fa...
Patricia C...
Josmercio ...
Luiza Mart...
Mogai Ti
Marcelo Pr...
Hudson Zanin
Ace Es
Tiago Amar...
Gabriel Va...
Emilio Bed...
Oscar Esga...
Tatiane Pi...
Jeane Alme...
Dulcinea P...
André Sant...
Fabiano Ruy
Victor Mar...
Daniel Rocco
Thiago Ras...
Felipe Sertã
Daniel Cug...
Renato Bes...
Paulo Bern...
Felipe Alves
Carlos Ale...
Laryssa Fl...
Ronaldo Lo...
Cristian S...
Fecatep ES
Mateuzin F...
Nina Fraga
Lukinha Ro...
Jefersson ...
Braian Car...
Rita Juliato
Bruno Card...
Miguel Di ...
Frederico ...
Vetinha Fr...
Helder Dias
Neumar Mal...
Fabiano Si...
Josmenio F...
Pedra Viva
Jeidi Siman
Transportation Networks

8. Complex Networks
• Share common characteristics - emergent behavior
• Formed with no centralized coordination
• Structure diverges from predictable topologies
(random, lattice)
8

9. Internet
9

10. PageRank
10

11. Complex Networks
• Formed with no centralized coordination
• Share common characteristics - emergent behavior
• Structure diverges from predictable topologies
(random, lattice)
• Non-trivial algorithms to capture network properties
11

12. Complex Network Research
standalone applications
12

13. Complex Network Research
script-based tasks
? code debug test result
13

14. Declarative querying
? query result
14

15. Objective
Databases
Com
plex!
Networks
15

16. Comparison
16

17. Explorative querying
17

18. Explorative querying
? query result
18

19. Workflow
19

20. Combining querying
and network analysis
Example: Point-of-Interest recommendation query
"retrieve 4 stars restaurants ranked by: geographic
proximity of a starting point and reputation of the
restaurant in a social network"
20

21. Query/analysis model
comparison
Query/analysis
model
Declarative
Graph
matching
Relational
aggregation
Graph
aggregation
Relational (SQL) ✅ ✅
Graph/CRPQs!
(sparql, cypher) ✅ ✅ ✅
Analysis frameworks!
(graphlab, graphx) ✅
21

22. Query/analysis model
comparison
Query/analysis
model
Declarative
Graph
matching
Relational
aggregation
Graph
aggregation
Relational (SQL) ✅ ✅
Graph/CRPQs!
(sparql, cypher) ✅ ✅ ✅
Analysis frameworks!
(graphlab, graphx) ✅
Beta Algebra (in*) ✅ ✅ ✅ ✅
22

23. Complex Data Management System
CDMS
in
Graph
Data
the query language allows correlation of
data based on combinations of topology-
aware measurements
the extended relational algebra can
express several flexible measurements
the declarative query combined with the
processing model allows new
mechanisms for query optimization
mappers facilitate relationship
manipulations and increase query flexibility
23

24. Complex Data Management System
CDMS
in
Graph
Data
the query language allows correlation of
data based on combinations of topology-
aware measurements
24

25. Combining querying
and network analysis
"Return nearby restaurants ranked by distance from
my position and reputation of the restaurant in a
social network"
25
in* query

26. Combining querying
and network analysis
"Return nearby restaurants ranked by distance from
my position and reputation of the restaurant in a
social network"
Traditional graph query (Cypher)
26

27. Combining querying
and network analysis
"Return nearby restaurants ranked by distance from
my position and reputation of the restaurant in a
social network"
Ranking based on a combination of
network measurements
27

28. Combining querying
and network analysis
"Return nearby restaurants ranked by distance from
my position and reputation of the restaurant in a
social network"
structured
predicates
topological
parameters
network
measurements
weight for
ranking
28

29. Combining querying
and network analysis
Geographic data, social networks, ontologies, text,
etc. can all be represented as networks and used in
the same query.29

30. Query examples
• retrieve candidate diagnosis relevant to a given
patient based on her symptoms and correlations
with other similar patients;
• retrieve documents related to the keyword query
"US elections" and the topic politics, written by
democrat journalists, ranked by relevance to the
keyword query and reputation of the author;
• retrieve fish species that play an important
(influential) role in the marine food web of a given
region and that are related to the biome 'coral reef'.
30

31. Network Measurements
"Return nearby restaurants ranked by distance from
my position and reputation of the restaurant in a
social network"
Ranking based on a combination of
network measurements
31

32. Complex Data Management System
CDMS
in
Graph
Data
the extended relational algebra can
express several flexible measurements
32

33. 33

34. 34

35. 35

36. 36

37. Query Model
• Declarative
• Combination of operators
• Relational aggregation
• Graph-based aggregation
37

38. Proposed Models
• Spreading Activation
• Relational Algebra
38

39. Query model
Query model Declarative
Graph
matching
Relational
aggregation
Graph
aggregation
Graph/CRPQs ✅ ✅ ✅
Spreading Activation ✅
Beta Algebra ✅ ✅ ✅ ✅
39

40. Beta-algebra: relational algebra
for query-based graph analysis
• Relational algebra with aggregation + beta
operator
• Beta operator is responsible for graph-based
aggregation
• Inspired by alpha-algebra (fixed point) [Agrawal 1988]
R. Agrawal. Alpha: An extension of relational algebra to express a class of recursive queries. 1988
40

41. Graph interpretation of
the beta operator
sub-operators responsible for data aggregation
along the traversal
41

42. Graph interpretation of
the beta operator
"Count paths from nodes a and l"
42

43. Graph interpretation of
the beta operator
43

44. Graph interpretation of
the beta operator
44

45. Graph interpretation of
the beta operator
45

46. Relational interpretation
of the beta operator
column
assignment join + column
assignment
group by +
aggregation
join + column
assignment
46

47. Relational interpretation
of the beta operator
column
assignment join + column
assignment
group by +
aggregation
join + column
assignment
47

48. Relational representation
of graphs
• Database contains Vertex and Edge relations
• Vertex relations in the form <id, a1, a2, … an>
• Edge relations in the form <id, idn, label, a1, a2, … an>
48

49. Beta operator
Beta(R, G, n, dir, follow, set_exp, map_exp, reduce_exp, update_exp)
:= update(update(..., reduce(...)), reduce(map(step(set(R)))))
simplified
execution plan
49

50. Beta operator
• A Beta operation is defined as a series of
relational sub-operations (implementation is
independent of the definition)
• Fixed number of repetitions (extension for other
stop conditions)
• Returns an edge relation
50

51. Beta operator
Beta(R, G, n, dir, follow, set_exp, map_exp, reduce_exp, update_exp)
• R - starting node table
• G - base graph (Vertex and Edge tables)
• n - number of steps
• dir - direction of the traversals
• follow - valid labels
• set_exp, etc - expressions for the sub-operators
51

52. Querying
"Count paths of length smaller than 3 from person
nodes with age greater than 30"
52

53. Querying
FIG. 6. Simplified execution tree template
1 set {count 1}
2 map {new.count current.count}
3 reduce ({count sum(count)},[id n])
4 update {current.count current.count + new.count}
5
6 R select(Person, {age > 30})
7 Count beta(R, G(Person,Knows), n:3, direction:BOTH, follow:[knows], set, map, reduce, update)
53

54. Querying
FIG. 6. Simplified execution tree template
1 set {count 1}
2 map {new.count current.count}
3 reduce ({count sum(count)},[id n])
4 update {current.count current.count + new.count}
5
6 R select(Person, {age > 30})
7 Count beta(R, G(Person,Knows), n:3, direction:BOTH, follow:[knows], set, map, reduce, update)
Starting
nodes
54

55. Querying
FIG. 6. Simplified execution tree template
1 set {count 1}
2 map {new.count current.count}
3 reduce ({count sum(count)},[id n])
4 update {current.count current.count + new.count}
5
6 R select(Person, {age > 30})
7 Count beta(R, G(Person,Knows), n:3, direction:BOTH, follow:[knows], set, map, reduce, update)
Aggregation
expressions
55

56. Querying
FIG. 6. Simplified execution tree template
1 set {count 1}
2 map {new.count current.count}
3 reduce ({count sum(count)},[id n])
4 update {current.count current.count + new.count}
5
6 R select(Person, {age > 30})
7 Count beta(R, G(Person,Knows), n:3, direction:BOTH, follow:[knows], set, map, reduce, update)
Traversal
options
56

57. PageRank-like queries
set={rank = 100}
map={new.rank = current.rank/c}
reduce=({rank = sum(rank)}, [id_n])
update= {current.rank = new.rank}
!
TableOrigins = select(V, {type == person})
TableRank = beta(TableOrigins, G, n:3, follow:[knows], set, map, reduce, update)
TableRank = order_by([rank], ASC)
"Find pagerank values for 3 iterations
over 'person' nodes and 'knows'
edges"
57

58. PageRank-like queries
58

59. PageRank-like queries
!
!
set={rank = prior}
map={new.rank = current.rank/c}
reduce=({rank = (1-beta)*sum(rank) + V.p*beta}, [id_n])
update= {current.rank = new.rank}
!
!
vPerson = TableUpdate(vPerson, {p = prior}, where: {id == 16 OR id == 14})
TableRank = beta(vPerson, G(E, vPerson), n:10, follow:[knows], set, map, reduce, update)
!
PageRank with priors
59

60. Distance-like queries
!
set =[{minDist = 0.0}, {maxDist = 0.0}]
map=[{minDist = minDist + Weight}, {maxDist = maxDist + Weight}]
reduce=([{minDist = min(minDist)}, {maxDist = max(maxDist)}], [id_n, id])
update=[{current.minDist = min(new.minDist, current.minDist)},
{current.maxDist = max(new.maxDist, current.maxDist)}]
!
TableOrigins = select(V, {id IN [3,4]})
TableDistances = beta(TableOrigins, G, n:4, follow:[connects], set, map, reduce, update)
TableDistances = select(TableDistances, {type == restaurant})
TableDistances = project(TableDistances, [id, id_n, minDist, maxDist])
TableDistances = order_by([id, id_n], ASC)
"Find minimum and
maximum distances from
points I3 and I4 to all
restaurants in at most 4
steps"
60

61. Distance-like queries
61

62. Complex Data Management System
CDMS
in
Graph
Data
the declarative query combined with the
processing model allows new
mechanisms for query optimization
62

63. Rank-preserving
optimizations (SA model)
• Retracing TSA
• Caching TSA
• Approximation
strategies
2 3 4 5
0
1
2
3
4
5
6
baseline
rTSA
cTSA
tuning of c
normalizedexecutiontime
63

64. Distance-like query
rewritings
“Distance between
the two nodes?”
64

65. start
65

66. start
66

67. Query rewriting
• Based on node degree (PageRank values tested)
• Heuristic: Always start from node of smaller degree
67

68. Distance-like query
rewritings
Ratio of total accessed nodes when starting from nodes of
lower degree (1000 queries per run)
68

69. Distance-like query
rewritings
Ratio of total accessed nodes when starting from nodes of
lower degree (1000 queries per run)
69

70. Complex Data Management System
CDMS
in
Graph
Data
mappers facilitate relationship
manipulations and increase query flexibility
70

71. Mappers
• Function like stored procedures for relationships
• Integrated in the query language
71

72. Mapper - projection
72

73. 73
Graph projection

74. 74
Graph projection

75. 75
Red. interact. with people
Rep. pain
Inab. restore energy even after sleep
Fatigue
Alter. ability continue prev. activit.
Alterat. depth breathing
Loss weight with adeq. food intake
Age 65 or over
Extrem. age
Chang. skin turgor
Imp. ability walk requir. distances
Decreas. lower extremity strength
Perceiv. limit. imposed by disease
Rep. inab. reach toilet in time avoid urine loss
Inab. delay defecation
Diarrhea
Evaluat. self unable deal sit./event.
Dysf. eat. patt.
Verb. dif. with prescribed regimens
Verb. rep. fatigue/weakness
Exertional discomfort
Exertional dyspnea
Sedent. activity level
Imbal. nutrit. st.
Weight 20% over ideal for height/frame
Moisture
Imp. sensation
Red. sensation
Chang. pigmentation
Inab. wash body
Seeks to be alone
Drowsy
Tachypnea
Dyspnea
Sleep disturb.
Facial tension
Insomnia Fatigue
Apprehensive
Anorexia
Dif. gait
Inab. dry body
Mak. choices inef. meet. health goals
Eat. resp. internal cues
Lack time meet person. needs
Uncert. regarding chang. relations. with care receiv.
Slowed movement
Limit. range motion
Dif. turning
Verb. exp. having no control
Exp. dissatisf. over inab. perform prev. activ.
Anger
Maternal-child interact. def.
Failure to thrive
Freq. accidents
Discomf. in social sit.
Chang. sleep patt.
Chang. patt.
Chang. availab. for affect. responsiv.
Chang. satisfaction with family
Inad. primary defenses
Trauma

76. Mapper
Xphone was
released today . . .
Comparing
XY.inc products . . .
Xphone vs
Yphone features . . .
Yphone
XY Inc
Xphone
features
Project
Employee
Alice
Bob
“Xphone features”
document indexing
entity
keyword
query
table record intermediate
node
76

77. Mapper
SELECT DISTINCT ?movie WHERE{
?movie a movie:film .
?movie movie:initial_release_date ?date .
FILTER ( fn:substring(?date, 1, 4) > "1990" ) .
FILTER EXISTS { ?movie foaf:page ?page }
}
RANKED BY
2 RELEVANCE OF ?movie TO
TokenMapper("My favorite films are Matrix and
Avatar")
DEPTH 2 FOLLOW TokenMapper:hasToken,
1 RELEVANCE OF ?movie TO movie:film_subject/461
DEPTH 5 FOLLOW skos:subject
77

78. Application of the
framework
• Databases/Complex Networks integration
• Application Integration (Information Retrieval,
Classification, Recommendation)
78

79. Exploratory Querying
79

80. Diagnosis
• Suggest diagnoses based on patient’s symptoms
• Data from the Nursing Department - Unicamp
• Case for query-based classification
80

81. 81
START diag=node(*)
WHERE has(diag.Type) and diag.Type = "Diagnose"
RETURN diag
RANK BY
2 RELEVANCE OF diag TO node( %p ) WEIGHTED,
1 CONNECTIVITY OF diag TO node( %p ) WEIGHTED
retrieve candidate diagnosis relevant to a given patient

82. Combining
measurements
"retrieve restaurants ranked by: geographic proximity of
a starting point, relevance to the keyword query 'family-
friendly restaurant', reputation in a social network, and
relevance to the semantic concept 'Casual Dining'"
Point-of-Interest recommendation query
82

83. Application Integration
Xphone was
released today . . .
Comparing
XY.inc products . . .
Xphone vs
Yphone features . . .
Yphone
XY Inc
Xphone
features
Project
Employee
Alice
Bob
“Xphone features”
document indexing
entity
keyword
query
table record intermediate
node
83

84. Query-based
application integration
Databases
Inform
ation
!
Retrieval
Complex!
Networks
Machine!
Learning
CDMS
84

85. Implementation
Neo4j
Open RDF Sesame
Virtuoso
Groovy
Gremlin
Blueprints
Java
Beta algebra (relational + graph)
85

86. Implementation
86

87. Publication
• Gomes Luiz, J. & Santanchè, A. (2015). The Web Within: Leveraging Web Standards and Graph Analysis
to Enable Application-Level Integration of Institutional Data. Transactions on Large-Scale Data- and
Knowledge-Centered Systems XIX, Springer Berlin Heidelberg, 2015, 26-54
• Gomes-Jr, L.; Amann, B. & Santanchè, A. (2015). Beta-Algebra: Towards a Relational Algebra for Graph
Analysis. GraphQ-EDBT, 2015
• Gomes-Jr, L. & Santanchè, A. (2014) Querying and Managing Complex Data. ADBIS PhD Consortium,
2014
• Gomes-Jr, L.; Jensen, R. & Santanchè, A. (2013). Towards query model integration: topology-aware, IR-
inspired metrics for declarative graph querying. GraphQ-EDBT 2013
• Gomes-Jr, L.; Jensen, R. & Santanchè, A. (2013). Query-based inferences in the Complex Data
Management System. Structured Learning: Inferring Graphs from Structured and Unstructured Inputs
(SLG-ICML), 2013
• Jensen, R.; Cruz, M.; Gomes-Jr, L. & Lopes, M. (2013) Attributing Fuzzy Values to Nursing Diagnoses and
Their Elements: The Specialists' Opinion. International Journal of Nursing Knowledge, 2013, 24, 134-141
87

88. Publication (in progress)
• Gomes-Jr, L.; Costa, L; Amann, B. & Santanchè, A. Integrating
Databases and Complex Networks
• Gomes-Jr, L.; Costa, L; Amann, B. & Santanchè, A. Beta-
algebra: Querying Complex Networks
88

89. Conclusion
• Proposed integration can benefit both areas
• Beta-algebra is able to capture diverse complex
network measurements
• Framework is applicable in several scenarios
• New query optimization techniques improve query
processing
89

90. Future Work
• Optimizations, distribution
• Redesign query language
• Study query-based learning models
• Improve support for text data
• Implement management mechanisms to support
graph warehouse
90

91. Luiz Gomes-Jr | gomesjr@ic.unicamp.br
thanks
obrigado
merci
gracias
ευχαριστώ
dank
mulțumesc
благодарам
CAPES
FAPESP
LIS
LIP6