3. Agenda
1. Faster and better software releases
2. Why JIRA for Quality Assurance?
3. Leverage JIRA for testing with Xray
I. Project Organization Scenarios
II. Workflows and Custom Fields
III. JQL functions
4. The life of a ”tester”
I. Specifying, organizing, planning, executing
II. Reporting
5. Test Automation
6. In sum
5. Speed vs Quality
Are they real enemies? Let these help you out:
• Agile
• DevOps
• Continuous Integration
• Automated Testing
• Continuous Testing
• Continuous Delivery
Release often, with confidence.
Motivate your team, make your customers happy.
Image by www.genalice.com.
6. Why is Test Management so important?
Software is everywhere, including in critical systems from health, finance, transports, space, IoT, ...
7. 2. Why JIRA for Quality Assurance?
Xray for JIRA
9. … there was Excel… or even other “legacy Test Management tools”
In the ”dark ages”….
10. Built-in features
• Focus on teams
• Collaboration / Comments
• Agile project management & waterfall
• Workflow control
• Customizable and adaptable
• Searching
• Assignment
• Track changes
• Linking between issues
• Importing issues from CSV
And more, through thousands of add-ons…
Welcome to “modern ages” with JIRA
11. What is missing?
1. Test related artifacts (tests, executions, plans)
2. Visibility of testing results and progress in Agile boards
3. In-context information about tests for requirements
4. Advanced reporting for generating documents and drill-down on data
JIRA as a Test Management tool
13. 1. Specify Pre-Conditions and Tests and organize them in Test Sets
2. Create one or more Test Plans for a given version
3. Create multiple planned Test Executions (i.e. associated with the Test Plan)
Dedicated JIRA issue types for testing
Specification: Tests and Pre-Conditions
Organization: Test Sets
Planning: Test Plans
Execution: (sub)Test Executions
14. 1. Leverages JIRAs built-in features
2. Permission control
3. Implementation of workflows
4. Custom fields
5. Screens customization
6. Assignment, Changes tracking
7. Collaboration
8. And much more…
The benefits of using JIRA issue types
19. Possible scenarios
• Disallow executions of Tests with workflow
statuses
• Disallow executions with workflow statuses
• For Test Executions in specific status
• In progress Test Execution workflow status
• After the first run
• Resolved Test Execution workflow status
• When all runs are in final status
• Resolution for Resolved Test Execution
Workflow example for Test Executions
Manage execution lifecyle
20. • Custom fields may be added to any Xray issue type (i.e. Test, Pre-Condition,
Test Set, Test Execution, Test Plan)
• Can be used to complement issues with additional information
Custom Fields
22. Querying using JQL functions
testPlanTests(‘CALC-123’)
Test Plan's Tests
Test Plan’s Tests currently in a given status
Test Plan’s Tests currently in a given status, in a given Test Environment
testPlanTests(‘CALC-123’, ’TODO’)
testPlanTests(‘CALC-123’, ’TODO’, ’Android’)
issue in requirements('NOK', 'Calculator', 'V4.0')
Requirement Issues with the provided coverage status
Xray provides an extensive list of JQL functions, so you may obtain what you need.
25. • Composed by a list of steps
• Each Step is defined by:
• Action/Step
• Data
• Expected Result
• Input Attachments
• Support for JIRA Wiki markup
• Importing steps, including from
Excel using copy&paste
Specifying “Manual Tests”
26. • Cucumber supported natively
• BDD – Specify tests in a business-readable domain-specific language
• Test definition serves as documentation
• Automated test steps may be implemented in Ruby, Java, .Net, etc
• Export specification and execute it during Continuous Integration
Specifying “Automated Cucumber Tests”
27. Astracts initial steps that must be done or ensured before starting the test.
• Manual or Automated
• Reusable; may be linked to multiple Tests
Complement specifications with “Pre-Conditions”
28. Flat, with “Test Sets”
Organizing Tests in multiple ways
A issue that represents a list of tests organized by some
logical way. The natural “JIRA way”.
For the “hierarchy/folder mindset”…
Hierarchical
29. A testing plan for aggregating and consolidate
results from multiple iterations (i.e. Test Executions)
• Tracks a group of tests and their results
independently of the number of executions
• Create planned test cycles (i.e. Test Executions)
for the Test Plan
Planning with “Test Plans”
30. • Contains a list of tests and their results (i.e. test runs)
• Can be labeled, proritized, assigned to components, etc.
• May be created manually or during Continuous Integration
A “task” for executing a group of tests on a given version of the system.
Executing with “Test Executions”
31. 1. Set Execution Result
2. Create Defect issues
3. Attach Evidences
4. Add comments
5. Check the Execution Activity Log
At the execution screen of a Test Run…
32. Ok… but how does gets visible in
our requirements?
Xray for JIRA
33. “Requirements” as they come by default in JIRA
• You may track the implementation through the workflow status but it lacks testing related info
• “Soft-binding” between implementation and expected result
34. Real status of requirements with in-context information
With Xray:
• Real-time information about the
coverage status of the requirement
for a given version
• Real-time information about the
associated test results, for a given
version
• A ”requirement” may be any issue
type you may have; test coverage
can be made in any issue type
36. 1. Built-in reports
2. JIRA gadgets in JIRA dashboards
3. Agile boards
4. Through other top-notch JIRA add-ons
Multiple ways of reporting, the JIRA way
37. Shows the requirement traceability, from requirements to defects.
Helps you quickly identify uncovered or incomplete/faulty requirements.
Built-in reports: Traceability Report
38. Provides a quick way of evaluating the current (coverage) status of your
project's requirements, so you will be able to see if they are ready to be or not.
JIRA Dasboards: Requirement Coverage Gadgets
39. Useful, yet lack quality related information.
Standard Agile Boards
40. Xray gives you visibility of your testing results and its progress.
Enhanced Agile Boards with Xray
41. Use Xporter to generate fully customizable Word, Excel or PDF files with your testing
reports to give to your management, colleagues or to your customers.
Reports as deliverable assets
47. 1. Use JIRA for managing the complete lifecycle of your software projects
2. Xray is the perfect tool for Test Management inside JIRA, for both manual and automated testing
3. Xporter complements Xray, enabling advanced reports in Word, Excel, PDF with custom layouts
Deliver faster and better releases with JIRA and Xray
Xray for JIRA empowers test teams to manage and deliver rock solid software solutions
48. w w w. x p a n d - a d d o n s . c o m
Thank you.
sergio.freire@xpand-it.com
Editor's Notes
Not really. You can achieve both if you adopt practices such as Agile, DevOps, Continuous Integration/Testing for example.
They can be used to ensure quality while allowing you release often.
Hoje em dia, o Software está em todo o lado e não pode falhar. Como tal, os testes são cada vez mais importantes para assegurar a qualidade do software.
Uma platafoma de Test Management é um software usado para gerir tests e execuções e produzir metricas e relatórios que permitam a visualização do estado dos Testes e Cobertura de Requisitos.
Temos aqui alguns exemplos de clientes nossos que usam os nossos add-ons para assegurar a qualidade dos seus produtos e serviços.
We often use JIRA for Requirement and Defect Management. It makes all sense to use it also for Test Management, because requirements, defects and tests are very close to each other.
Many organizations still use Excel or Word or other things to do test specification and reporting respective results…
This does is very rudimentary and does not scale.
JIRA has many built-in features that are great for Test Management purposes, such as all the collaboration related features, issue assignment and searching, linking between issues, change tracking and many other ones.
Pros
Self-contained
Easy to understand and manage
Promotes team collaboration
Cons
You may not want to have Bugs, Test Executions in the same project for security concerns
This separation allows to manage permissions more effectively, so maybe only a set of testers has permission to write tests and others to execute.If you already have a JIRA project for managing requirements and defects and don't want to have any tests nor executions in this project, you can create a separate companion project just for testing purposes.
Pros
This separation allows to manage permissions more effectively, so maybe only a set of testers has permission to write tests and others to execute
Keeps your existing requirements and defects project as-is, completely separated from testing related tasks
Isolates Test specification and organization
Keeps sensible data related with bugs and test planning/execution in a different project
Cons
Requires manually synchronizing versions from the requirements project to the other ones
More projects to manage
May promote team separation
All results related with a given test specification are always reported to the same Test entity
Runs history, comments, etc, all will be in the same entity
Tests may be added to Test Sets, Test Plans and may be part of Test Executions
Tests may be linked with requirements
test results will be reflected on requirements