QAestro relies on a controlled vocabulary and the Local-as-View (LAV) approach to model QA tasks and components, respectively. We have formalized 51 existing QA components implemented in 20 QA systems using QAestro. Our empirical results suggest that QAestro enumerates the combinations of QA components that effectively implement QA developer requirements. Published in Proceedings of DEXA 2017, Lyon, France

1. QAestro - Semantic-based Composition of
Question Answering Pipeline
DEXA 2017
28.08.2017
Kuldeep Singh, Ioanna Lytra, Maria Esther Vidal, Dharmen Punjani,
Harsh Thakkar, Christoph Lange, Sören Auer

2. Motivating Example
● Question: “Where was Albert Einstein born?”
● Query in a Formal Language: e.g. “SPARQL Query”
2

3. Motivating Example
● Question: “Where was Albert Einstein born?”
● Query in a Formal Language: e.g. “SPARQL Query”
3

4. Motivating Example
4
Question Answering Tasks

5. ● More than 70 QA
systems in last 5
years.
● QA systems
implement similar
tasks to answer
user’s question.
5
Motivating Example

6. Challenge
6
OKBQA Question Answering Framework

7. Problem Statement
● Define a framework able to
○ semantically describe QA components and QA developer
requirements; and
○ produce QA component compositions based on these
semantic descriptions.
7

8. Our Contributions
● The QAestro framework that generates QA component
compositions based on developer requirements.
● A vocabulary for expressing QA tasks and developer
requirements.
● The formalization of existing 51 QA components from 20 QA
systems.
● A mapping of the QA component composition problem into
Query Rewriting Problem (QRP).
● An empirical evaluation of QAestro framework.
8

9. Agenda
1. Approach
a. Controlled Vocabulary for Abstract QA Tasks
b. Semantic Descriptions of QA Component
c. QA Developer Requirement
d. The Problem of QA Component Composition
2. Empirical Study
3. Conclusions
9

10. Approach

11. Controlled Vocabulary for Abstract QA Tasks
● QAV= (𝛿,A)
○ 𝛿: signature of logical language.
○ A: set of axioms describing the relationships among vocabulary
concepts.
11

12. Controlled Vocabulary for Abstract QA Tasks
● QAV= (𝛿,A)
○ 𝛿: signature of logical language.
○ A: set of axioms describing the relationships among vocabulary
concepts.
● Example: Disambiguation(x, y, z, t) → QuestionAnalysis(x, y)
○ where x=entity; y=question; z=disambiguated entities, t=template ;
12

13. Semantic Description of QA Component
● Follow Local as View (LAV) approach to define QA components.
○ Allows easy scaling up to large number of QA components.
● Example:
○ Describe Stanford NER that performs named entity recognition
task.
13

14. Semantic Description of QA Component
● Example:
○ StanfordNER($y,x) :– Recognition(y,x), Question(y), Entity(x)
■ Recognition is the QA task of Stanford NER component.
■ StanfordNER accepts question as input and gives recognised
entities as output.
■ Question, Entity, variables x and y are part of controlled
vocabulary.
14

15. QA Developer Requirement
● Example: Give me all the components that implements entity
recognition and entity disambiguation task.
● Semantic Description of QA Developer Requirement:
○ QADevReq($y, x) :– Recognition(y,x), Disambiguation(x,y,z,t)
15

16. The Problem of QA Component Composition
● How to compose valid compositions using LAV mappings of QA
component.
LAV Mapping Examples:
○ DBpediaNER($y, x) :– Recognition(y, x), Question(y), Entity(x)
○ Alchemy($y, z) :– Disambiguation(x, y, z, t), Question(y), DisEntity(z)
○ Qakisatype($y, a) :– Answertype(y, a, o), Question(y), Atype(a)
○ StanfordNER($y,x) :– Recognition(y,x), Question(y), Entity(x)
○ Agdistis($x, $y, z) :– Disambiguation(x, y, z, t), Entity(x), Question(y),
disEntity(z)
16

17. 17
QAestro Framework

18. QA Component Composition using QAestro
● QADevReq($y, x) :– Recognition(y, x), Disambiguation(x, y, z, t),
answertype(y, a, o)
Results:
○ QADevReq($y, x) :– StanfordNER($y, x), Agdistis($y, $x, z),
Qakisatype($y, a)
○ QADevReq($q, e) :– DBpediaNER($y, x), Agdistis($y, $x, z),
Qakisatype($y, a)
18

19. Empirical Study

20. Experiment Configuration
● QAestro Implementation
○ Implemented in python 2.7 on top of MCDSAT.
○ Source Code: https://github.com/WDAqua/QAestro
○ 51 QA component formalisation for 20 QA systems
● QAestro Experiments
○ Executed on a laptop (with Fedora Linux 25) Intel i7-4550U,
4x1.50GHz and 8GB RAM.
20

21. Analysis of QA Components
21

22. Analysis of QA Components
22

23. QA Component Composition
23

24. QA Component Composition
24

25. Conclusions
● Introduce QAestro- a framework that:
○ Semantically describe QA Components and Developer
Requirements.
○ Follows Local as View approach.
○ Compose valid compositions of QA components.
○ Can successfully deal with the growing number of QA.
○ Demonstrates efficient processing time.
25

26. Find us!
● For more information please visit: http://wdaqua.eu/QAestro/
● Demo can be viewed at:
○ https://www.youtube.com/watch?v=9lhamebx7JM&feature
=youtu.be
● Write me on : kuldeep.singh@iais.fraunhofer.de
26