Talk at EMBO Workshop, Network inference in biology and disease, 10–13 September 2019 TIGEM - Naples, Italy

1. EMBO
Workshop
Network inference in
biology and disease
10–13 September 2019
TIGEM - Naples, Italy
Continuous time Bayesian networks to infer
global regulatory networks in humans
Fabio Stella1 and Teresa Zelante2
1 University of Milan-Bicocca
Department of Informatics, Systems and Communication
2 University of Perugia
Dipartimento di Medicina Sperimentale

2. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
1/22

3. SUMMARY
▪ Regulatory networks
▪ BAYESIAN NETWORKS
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
A
B
C
D
E
F
G
1/22

4. SUMMARY
▪ Regulatory networks
▪ BAYESIAN NETWORKS
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
A
B
C
D
E
F
G
C ∈ 𝑐1, 𝑐2
F ∈ 𝑓1, 𝑓2E ∈ 𝑒1, 𝑒2
C
E e 1 e 2 e 1 e 2
f 1 0.30 0.90 0.40 0.75
f 2 0.70 0.10 0.60 0.25
c 1 c 2
F
2/22

5. SUMMARY
▪ Regulatory networks
▪ BAYESIAN NETWORKS
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
Friedmanetal.(2000)
3/22

6. 𝑄 𝐴
𝑐1
=
−𝜆 𝑎1→
𝑐1
𝜆 𝑎1→𝑎2
𝑐1
𝜆 𝑎1→𝑎3
𝑐1
𝜆 𝑎2→𝑎1
𝑐1
−𝜆 𝑎2→
𝑐1
𝜆 𝑎2→𝑎3
𝑐1
𝜆 𝑎3→𝑎1
𝑐1
𝜆 𝑎3→𝑎2
𝑐1
−𝜆 𝑎3→
𝑐1
=
−0.03 0.02 0.01
5.99 −6.0 0.01
1.00 5.00 −6.0
SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
A ∈ 𝑎1, 𝑎2, 𝑎3
B ∈ 𝑏1, 𝑏2
C ∈ 𝑐1, 𝑐2
rate at which A leaves state 𝑎3
when it’s parent C is fixed to
state 𝑐1. (Exp. distribution)
Probability A to transition
from state 𝑎3 to state 𝑎2.
𝜆 𝑎3→𝑎2
𝑐1
𝜆 𝑎3→
𝑐1
=
5.00
6.00
= 0.83
allows
cycles
4/22

7. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
5/22

8. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
=
=
6/22

9. Dialogue for Reverse Engineering Assessments and Methods
Subnetworks from the known in vivo gene regulatory network structures of
E. coli and S. cerevisiae were extracted, endowing them with dynamic
models of regulation including transcriptional and translational processes.
Subnetworks: 10, 20, 50 and 100 genes randomly extracted.
Robustness: 10 networks for each gene set.
▪ Wild type time course data (unperturbed network).
▪ Interventional time course data including
• extensive knockouts
• knockdowns
• multifactorial perturbations
• (when applicable) dual knockouts
GeneNetWeaver default
• 21 time points
• 10 replicates
SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ SYNTHETIC DATA
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
7/22

10. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ SYNTHETIC DATA
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
8/22

11. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ SYNTHETIC DATA
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
9/22

12. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ SACCAROMYCES
CEREVISIAE
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
Cantoneetal.(2009)
10/22

13. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ SACCAROMYCES
CEREVISIAE
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
Precision Recall F1
GC 0.80 0.50 0.62
DBNs 0.75 0.38 0.50
CTBNs 0.83 0.63 0.71
Cantoneetal.(2009)
10/22

14. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T HELPER 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
T helper 17 (TH17) cells
▪ represent a pivotal adaptive cell subset involved in multiple immune
disorders in mammalian species.
▪ deciphering the molecular interactions regulating TH17 cell
differentiation is particularly critical for novel drug target discovery
designed to control maladaptive inflammatory conditions.
Goal
▪ Infer the global regulatory network controlling TH17 differentiation by
using time-course gene expression data.
11/22

15. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T HELPER 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
12/22

16. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ CONTINUOUS TIME
BAYESIAN NETWORKS
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T HELPER 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
Results
▪ we identified the Prdm1 gene encoding the B lymphocyte-induced
maturation protein 1 as a crucial negative regulator of human TH17 cell
differentiation.
▪ results have been validated by perturbing Prdm1 expression on freshly
isolated CD4+ naïve T cells: reduction of Prdm1 expression leads to
augmentation of IL-17 release.
▪ these data unravel a possible novel target to control TH17 polarization in
inflammatory disorders.
13/22

17. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
14/22

18. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
14/22

19. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
15/22

20. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
15/22

21. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ DROSOPHILA
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
16/22

22. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ Drosophila
✓ SACCAROMYCES
CEREVISIAE
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ Conclusions
17/22

23. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ SONGBIRD
▪ Adding memory
▪ Causal networks
▪ Conclusions
18/22

24. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ NON-STATIONARY
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ SONGBIRD
▪ Adding memory
▪ Causal networks
▪ Conclusions
18/22

25. • Erlang
• Erlang-Coxian
• Mixture
▪ Hidden nodes bring memory
▪ Phase-type distributions (universal approximators)
SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ ADDING MEMORY
▪ Causal networks
▪ Conclusions
19/22

26. A
B
The conditional probability of B at
time «s+t» given the state of node
A is known from time «s» to time
«s+t» is equal to the conditional
probability of B at time «r+t» given
the state of node A is known from
time «r» to time «r+t».
The process associated with B in
memoryless!!!
SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ ADDING MEMORY
▪ Causal networks
▪ Conclusions
𝑃𝐵|𝐴 𝑠 + 𝑡|𝑠 = 𝑃𝐵|𝐴 𝑟 + 𝑡|𝑟
20/22

27. h
h provides memory to node B
SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ ADDING MEMORY
▪ Causal networks
▪ Conclusions
𝑃𝐵|𝐴 𝑠 + 𝑡|𝑠 ≠ 𝑃𝐵|𝐴 𝑟 + 𝑡|𝑟
A
B
21/22

28. SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ CAUSAL NETWORKS
▪ Conclusions
CHOLESTEROLEXERCISE
AGE
22/2222/22

29. Continuous time Bayesian networks
▪ allow efficient learning from data
▪ offer great expressiveness (their extensions)
▪ achieve performance comparable to state-of-the-art
▪ can be interpreted and explained by domain experts
Machine Learning, Artificial Intelligence and Data Science
▪ data alone can not answer many relevant questions
▪ to make effective decisions we need causal knowledge
▪ we need to consider the story behind the data
▪ causal data science from observational data is the new challenge
SUMMARY
▪ Regulatory networks
▪ Bayesian networks
▪ Continuous time
Bayesian networks
✓ Synthetic data
✓ Saccaromyces
Cerevisiae
✓ T Helper 17
▪ Non-stationary
✓ Drosophila
✓ Saccaromyces
Cerevisiae
✓ Songbird
▪ Adding memory
▪ Causal networks
▪ CONCLUSIONS

30. EMBO Workshop
Network inference in
biology and disease
10–13 September 2019
TIGEM - Naples, Italy
Special Issue on
Machine Learning and Big Data for Disease and Health
Time Schedule
Expression of Interests deadline: October 13, 2019
Manuscript submission deadline: February 28, 2020
Manuscript acceptance notification: June 30, 2020
Tentative Publication Date: September 2020

31. References
Acerbi, E., Viganò, E., Poidinger, M., Mortellaro, A., Zelante, T., Stella, F. (2016). “Continuous time Bayesian networks identify Prdm1
as a negative regulator of TH17 cell differentiation in humans”. Scientific Reports, 6, 23128.
Acerbi, E., Zelante, T., Narang V., Stella, F. (2014). “Gene network inference using continuous time Bayesian networks: a comparative
study and application to Th17 cell differentiation”. BMC Bioinformatics. 15:(387), doi:10.1186/s12859-014-0387-x, ISSN: 1471-2105.
Cantone, I., Marucci, L., Iorio, F., Ricci, M.A., Belcastro, V., Bansal, M., Santini, S., di Bernardo, M., di Bernardo, D., Cosma, M.P.
(2009) “A yeast synthetic network for in vivo assessment of reverse-engineering and modeling approaches”. Cell 137, 172–181.
Dondelinger, F., Lebre, S., & Husmeier, D. (2013). Non-homogeneous dynamic Bayesian networks with bayesian regularization for
inferring gene regulatory networks with gradually time-varying structure. Machine Learning, 90 (2), 191-230.
Friedman, N., Linial, M., Nachman, I., and Pe’er, D. (2000). Using Bayesian networks to analyze expression data. In Proceedings of the
fourth annual international conference on Computational molecular biology (RECOMB '00). ACM, New York, NY, USA, 127-135.
Lebre, S., Becq, J., Devaux, F., Stumpf, M., & Lelandais, G. (2010). Statistical inference of the time-varying structure of gene regulation
networks. BMC Systems Biology, 4 (1), 130+.
Liu, M., Stella, F., Hommersom, A., Lucas, P. (2018). “Making Continuous Time Bayesian Networks More Flexible”. Proceeding of
Machine Learning Research.
Dean, T., & Kanazawa, K. (1989). A model for reasoning about persistence and causation. Comput. Intell., 5 (3), 142-150.
Nodelman, U., Shelton, C., & Koller, D. (2003). Learning continuous time bayesian networks. In The 19th Conference on Uncertainty in
Artificial Intelligence (UAI 2003), Acapulco, Mexico, pp. 451-458.
Pearl, J. (1989). Probabilistic reasoning in intelligent systems - networks of plausible inference. Morgan Kaufmann series in representation
and reasoning. Morgan Kaufmann.
Robinson, J. W., & Hartemink, A. J. (2010). Learning non-stationary dynamic Bayesian networks. Journal of Machine Learning Research,
11, 3647-3680.
Villa, S., Stella, F. (2016). “Learning Continuous Time Bayesian Networks in Non-stationary Domains”. Journal of Artificial Intelligence
Research, 57:1, pp. 1-37.