ElasTest is an open source project aimed at easing end-to-end testing for large complex distributed systems. This presentations gives an overview of the main ElasTest features, the underlying architecture and the main components.

1. http://elastest.io
ElasTest
Funded by the
European Union
Why, What, Who,
Where, When, and How
ElasTest Overview Presentation | 24/05/17 | Madrid URJC

2. Why?
2

3. Why ElasTest?
●
Modern systems are complex
3
Distributed Software / Cloud Native / Microservices

4. Why ElasTest?
●
We need
tests to
assure
their
quality
4

5. Why ElasTest?
5
What tools do exist for E2E testing in
complex cloud native apps?

6. Why ElasTest?
●
The IDE (to develop tests)
6

7. Why ElasTest?
●
The CI Server (to run tests after changes)
7

8. Why ElasTest?
●
These tools are focused in managing test
execution
8

9. Why ElasTest?
●
These tools are focused in managing test
execution
9

10. Why ElasTest?
●
These tools are focused in managing test
execution
10

11. Why ElasTest?
●
Current testing tools have limitations
when facing E2E tests of complex
distributed software
– Log management
– Test orchestration
– Comparing test executions
– Web testing
– SuT instrumentation
11

12. Limitations of “testing tools”
●
Log management
12

13. Limitations of “testing tools”
●
Log management
13
Where in the
source code is the
log of the failing
test?

14. Limitations of “testing tools”
●
Log management
– How to query logs of the System Under
Test?
14
Elasticsearch Kibana Logstash

15. Limitations of “testing tools”
15

16. Limitations of “testing tools”
16
Test logs are in the
IDE / CI, not here

17. Limitations of “testing tools”
17
Which log entries are
related to failure?
Time filter?

18. Limitations of “testing tools”
18
Is that the best tool to
know why a test has
failed?

19. Limitations of “testing tools”
19
OtrosLogViewer is a powerful log analysis tool

20. Limitations of “testing tools”
●
What if a test wants to know if there are
WARN messages in SuT logs?
20

21. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log management
– Test orchestration
– Comparing test executions
– Web testing
– SuT instrumentation
21

22. Limitations of “testing tools”
●
Test orchestration
– Usually we want to execute the same test, but
with different:
●
Input data, browser version, SuT
configuration...
– CI Tools are improving in this domain...
●
Jenkins pipelines
●
Travis Build Stages
●
Cross browser testing services
22

23. Limitations of “testing tools”
●
Jenkins pipelines
23

24. Limitations of “testing tools”
●
Travis Build Stages
24
https://blog.travis-ci.com/2017-05-11-introducing-build-stages

25. Limitations of “testing tools”
●
Cross browser testing services
25

26. Limitations of “testing tools”
●
Some use cases are still complex

27. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log management
– Test orchestration
– Comparing test executions
– Web testing
– SuT instrumentation
27

28. Limitations of “testing tools”
●
Comparing test executions
– When there’s a regression, one compares the
failing test log with a previous correct
execution log looking at the differences
– CI Tools only show test management info...
28

29. Limitations of “testing tools”
●
Diff tools are not designed to compare logs
29

30. Limitations of “testing tools”
●
JMeter doesn’t compare several executions
30

31. Limitations of “testing tools”
●
How we create graph comparisons?
31http://bravenewgeek.com/dissecting-message-queues/
Latency in several message queues

32. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log management
– Test orchestration
– Comparing test executions
– Web testing
– SuT instrumentation
32

33. Limitations of “testing tools”
●
Web Testing
– Selenium is used to control
browsers
– There are problems when
developing tests
●
Browsers are opened during
execution grabbing the focus.
●
You can’t work while tests are
running in your laptop
33

34. Limitations of “testing tools”
●
Web Testing
– There are problems when
executing tests in CI
●
You have to configure Xvfb for
headless testing
●
You have to configure custom tools
to record test executions
●
Video files are not synchronized with
logs (you have to jump manually)
34

35. Limitations of “testing tools”
●
Web Testing
– Docker is helping here
●
Official selenium
images
– However
●
A lot of manual
configuration needed
●
No solution for Safari
or Edge
35

36. Limitations of “testing tools”
●
Web Testing
– Commercial cloud services solve
these problems
36

37. Limitations of “testing tools”
●
Web Testing
– Testing WebRTC (video) apps
requires new features
●
Quality of Service (QoS)
●
Quality of Experience (QoE)
●
Real-time Latency
●
Testing Network Topologies (ICE)
37

38. Why ElasTest?
●
Current testing tools have limitations
with E2E tests of complex distributed
software
– Log management
– Test orchestration
– Comparing test executions
– Web testing
– SuT instrumentation
38

39. Why ElasTest?
●
SuT instrumentation
– Fault tolerance / Resilience testing
– Testing with different network conditions
– Execute the same test with different SuT
conditions (orchestration)
39

40. Why ElasTest?
40

41. Why ElasTest?
●
SuT instrumentation
– No integration with CI systems
– Hard to manage that from tests
– Hard to compare test executions with
different SuT configurations
41

42. What is?
42

43. What is ElasTest?
●
An open source tool
●
Designed to ease E2E testing of
complex distributed systems
●
You can install it in your laptop and in
your CI server
43
Apache License 2

44. What is ElasTest?
●
How ElasTest is controlled?
– Through a web interface (Jenkins like)
– From CI servers plugins
– From IDE plugins (like plugins to deploy
apps in remote servers)
– From a command line tool (Maven like)
44

45. What is ElasTest?
●
How do you develop your tests?
– With your favorite language
– With your favorite testing framework
– With your usual tools like selenium
– There are libraries for Java and
JavaScript to ease development
45

46. Who is creating?
46

47. Who is creating ElasTest?
●
ElasTest is developed as a project
funded by the European Commission
47
http://elastest.eu
ElasTest: an elastic platform for testing
complex distributed large software systems
Duration: From 2017-01-01 to 2019-12-31
Total cost: EUR 5 031 187,50
EU contribution: EUR 4 270 187
Coordinated in: Spain
Topic(s): ICT-10-2016 - Software Technologies
Call for proposal: H2020-ICT-2016-1
Funded by the European Union

48. Who is creating ElasTest?
●
Project coordinator
48

49. Who is creating ElasTest?
●
Participant institutions / companies
49

50. Who is creating ElasTest?
●
We at CodeURJC have
experience in testing
complex distributed
systems
50

51. Where is?
51

52. Where is ElasTest?
●
ElasTest is developed in GitHub
●
There are several communication channels
52
http://elastest.io
https://github.com/elastest

53. When will be available?
53

54. When ElasTest will be available?
●
Roadmap
54
Milestone Title Date
M1 First ElasTest architecture April 2017
M2 First ElasTest artifacts August 2017
M3 Early ElasTest platform December 2017
M4 ElasTest CI system April 2018
M5 Mature ElasTest platform December 2018
M6 ElasTest fully qualified December 2019

55. How?
55

56. How? ElasTest architecture
56

57. How? ElasTest architecture
●
ElasTest is a microservice application
●
Made up of several components, talking to
each other via
– REST APIs
– Message queues
57

58. How? ElasTest architecture
●
Test orchestration and recommendation
manager (TORM)
– Entry point of ElasTest Platform
– Provides API and Web UI for:
●
Test management
●
Test execution
●
Test orchestration
●
Test comparison
58

59. How? ElasTest architecture
●
Platform Manager (EPM)
– Abstracts other components from the
underlying infrastructure
– Provides resources for the other components
– Supported platforms
●
Public clouds (AWS)
●
On premise (OpenStack, Kubernetes)
●
Also in your laptop (Docker)
59

60. How? ElasTest architecture
●
Data Manager (EDM)
– Provides persistence for other components
●
Management data (MySQL)
●
Logs & metrics (ElasticSearch)
●
Files (Alluxio)
60

61. How? ElasTest architecture
●
Intrumentation Manager (EIM)
– Instrument Software under Test (SuT) to:
●
Gather logs and metrics
●
Simulate real network conditions (low
bandwith, high latency, connectivity loss…)
●
Force node failures
●
Simulate node loads with CPU throttling
61

62. How? ElasTest architecture
●
Test engines
– Engines complement the functionalities of
ElasTest with services to help in test definitions
●
Orchestrator Engine (EOE): Orchestrate and
compare several tests
●
Cost Engine (ECE): estimates costs of running a
given test suite on a given provider
●
Recommendation Engine (ERE): recommends
additional tests for similar contexts
●
Question & Answer Engine (EQE): Q&A UI for
gathering additional information about tests
62

63. How? ElasTest architecture
●
Test services
– Tests need to use some services in E2E
testing
●
Browsers
●
Mobile emulators
●
IoT (devices/actuators/sensors) emulators
●
Security services
●
Big Data services
63

64. How? ElasTest architecture
●
Service Manager (ESM)
– A registry for services within the ElasTest
platform
●
OSBA compliant
– Monitors service health status
– Makes ElasTest extensible
64

65. How? ElasTest architecture
●
Built-in ElasTest services
– User impersonation Service (EUS): Provides
browsers and mobile emulators for tests
– Device impersonation Service (EDS): Provides
sensors, actuators and devices emulators for tests
– Security Service (ESS): Provides security checks
for tests
– Big Data Service (EBD): Big data is available for
tests for runtime analysis
– Monitoring Service (EMS): Provides test execution
metrics and logs
65

66. Do you have any question?
66
Thanks!
info@elastest.io