This talk was given during the Asia Agile Test Alliance Conference 2015. ThiIn this presentation the Agile test practices were mapped to the continuous delivery practices. The presentation provides quick view about the broad anti patterns and then provide the key solution by following the Onion Layered Test Practice Map (Task, Story, Sprint, Release Level). It is well known that without good test practice and quality test, continuous delivery may only give false confidence.

1. Confidential | Copyright © Agile Testing Alliance Global Gathering
Brought To You By
&
Topic: Onion Layered Agile Test Practice Map
Presenter: Anish Cheriyan, Director Quality and CoE-Software Engineering & Test , Huawei
Contributed by: Rajith R- Director, Sriharsha N- SeniorTest Architect- CoE-Software Engineering & Test

2. Confidential | Copyright © Agile Testing Alliance Global Gathering
Topic Coverage
• Test Anti Patterns for Continuous Delivery
• Agile Test Practices Mapped
– Task
– Story
– Sprint
– Release
• Examples of some Anti Patterns and Corrections
• Conclusion

3. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by Naomi King - Creative Commons Attribution License https://www.flickr.com/photos/22141297@N06 Created with Haiku Deck
Continuous Integration
Continuous Delivery
Continuous Deployment
DevOps

4. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by fr1zz - Creative Commons Attribution-NonCommercial-ShareAlike License https://www.flickr.com/photos/39143000@N00 Created with Haiku Deck

5. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by Paul Stevenson - Creative Commons Attribution License https://www.flickr.com/photos/53496815@N00 Created with Haiku Deck

6. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by Mark Fischer - Creative Commons Attribution-ShareAlike License https://www.flickr.com/photos/80854685@N08 Created with Haiku Deck

7. Confidential | Copyright © Agile Testing Alliance Global Gathering
INVERTED TEST PYRAMID
Picture Courtesy: blogs.agilefaqs.com

8. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by Stéfan - Creative Commons Attribution-NonCommercial-ShareAlike License https://www.flickr.com/photos/49462908@N00 Created with Haiku Deck
Initial+*++++++++

9. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by Enzymlogic - Creative Commons Attribution-ShareAlike License https://www.flickr.com/photos/101755654@N08 Created with Haiku Deck

10. Confidential | Copyright © Agile Testing Alliance Global Gathering
Com
mit
UT &
Inspe
ction
Functi
onal
Build
UAT …
DEPLOYMENT PIPELINE
ORCHESTRATION NOT WELL THOUGHT
OF..

11. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by tunnelarmr - Creative Commons Attribution-NonCommercial-ShareAlike License https://www.flickr.com/photos/27311060@N00 Created with Haiku Deck
FOCUS ONLY ON
TOOLS/FRAMEWORKS NOT ON
AUTOMATION ARCHITECTURE

12. Confidential | Copyright © Agile Testing Alliance Global GatheringPhoto by nick farnhill - Creative Commons Attribution-ShareAlike License https://www.flickr.com/photos/57749032@N00 Created with Haiku Deck

13. Confidential | Copyright © Agile Testing Alliance Global Gathering
INFRASTRUCTURE
CODE NOT TESTED

14. Confidential | Copyright © Agile Testing Alliance Global Gathering
NO FOCUS ON
HARDENING

15. Confidential | Copyright © Agile Testing Alliance Global Gathering
Test Practices Mapped
Read at : http://www.thinkinginagile.com/2015/07/agile-testing-practices-mapped-to.html

16. Confidential | Copyright © Agile Testing Alliance Global Gathering
Test Automation Strategy
Picture Courtesy: Agile Testing-Lisa Crispin, Janet Gregory

17. Confidential | Copyright © Agile Testing Alliance Global Gathering
Example- Test Automation Architecture(Anti Pattern) and Corrected
Client (GUI)
Platform Server
Data Access Layer
Database
System Under Test

18. Confidential | Copyright © Agile Testing Alliance Global Gathering
Example- Test Automation Architecture(Anti Pattern) and Corrected
GUI Automated using QTPro
System Under Test
Initial Automation Framework
Initial

19. Confidential | Copyright © Agile Testing Alliance Global Gathering
Example- Test Automation Architecture(Anti Pattern) and Corrected
Fire the test SuiteInit Layer
Adaptation thru GUI, API
Adapter Layer
Utility Layer System Under TestUtility Layer
Reporting as HTML, Error log..Reporting Layer
Updated Automation Architecture
Corrected

20. Layered Automation Framework –A simple Case
Start
Step a
Step b
Step c
Step d
Step e
Step f
Step g
Step h
Step i
Step j
End
Test Cases Before Layering
Change in any small functionality can
impact steps and as an impact all Test
Cases have to be re-worked to adapt to
new change – Result : Huge Rework
effort and poor maintainability
Impact Of functional change on Test cases
before Layering
Start
Step a
Step b
Step x
Step d
Step y
Step z
Step g
Step h
Step i
Step j
End
Start
Step a
Step b
Step c
Step d
Step p
Step q
Step r
Step g
Step h
Step i
Step j
End
Step a
Step b
Step x
/Step c
Step d
Function 1
(input = Type x/c)
Step g
Step h
Step I
Step j
Function 2()
Start
Function1(c)
Step e
Step f
Function2()
End
Test Cases after Functional Layering
Most Change in functionality can be
reworked at functional or business layer
level – Result : Very Less Rework effort,
High maintainability and Reusability
Impact Of functional change on Test cases
after Layering
Start
Function1(x)
Step y
Step z
Function2()
End
Start
Function1(c)
Step p
Step q
Step r
Function2()
End
Start
Buss Func 1(e,f)
End
Start
Buss Func 1(y,z)
End
Start
Buss Func 1(p,q,r)
End
Test Cases after Business Layering
Buss Func 1(input = Steps)
• Function 1
(input = Type x/c)
• Steps based on input
• Function 2()

21. Confidential | Copyright © Agile Testing Alliance Global Gathering
Example- Test Suite Organization (Anti Pattern) and Corrected
SendEmail_UT_1
SendEmail_UT_2
SendEmail_UT_3
SendEmail_UT_4
SendEmail_UT_5
SendSMS_UT_1
SendSMS_UT_2
SendSMS_FuncTest_1
SendSMS_FuncTest_2
SendSMS_FuncTest_3
SendSMS_FuncTest_4
SendSMS_ReliabilityTest 1
SendSMS_ReliabilityTest 2
SendSMS_ReliabilityTest 3
SendEmail_FuncTest_1
SendEmail_FuncTest_2
SendEmail_FuncTest_3
SendEmail_FuncTest_4
SendEmail_FuncTest_5
SendEmail_PerfTest_1
SendEmail_PerfTest_2
SendSMS_UT
Suite
•SendSMS_UT_1
•SendSMS_UT_2…
SendSMS_Func
Test Suite
•SendSMS_FuncTes
t_1
•SendSMS_FuncTes
t_2
•SendSMS_FuncTes
t_3..
SendSMS_Reliabilit
TestSuite
•SendSMS_Reliabi
lityTest 1
•SendSMS_Reliabi
lityTest 2
•SendSMS_Reliabi
lityTest 3..
Send SMS
Feature
Suite
Test Case
SendEmail_UT
Suite
•SendEmail_UT_1
•SendEmail_UT_2
•SendEmail_UT_3…
SendEmail_FuncTe
st Suite
•SendSMS_FuncTes
t_1
•SendSMS_FuncTes
t_2
•SendSMS_FuncTes
t_3..
SendEmail_PerfTes
t Suite
•SendEmail_PerfTe
st_1
•SendEmail_PerfTe
st_2..
Send Email
Feature
Suite
Test Case
Initial Corrected

22. Confidential | Copyright © Agile Testing Alliance Global Gathering
Example- Initial Deployment Pipeline(Anti Pattern) and Corrected
Commit
Inspectio
n
Function
al Build
‘Ities’
Test
…
CI Dev Build
Image Courtesy: http://geekswithblogs.net/TarunArora/archive/2012/09/18/vs-2012-code-review-ndash-before-check-in-or-after.aspx
Manual Review
Manual Test and ReleaseCI Test Build
Private Build and Expert Review
Dev and Test Case Organized, Quality Parameters Built-in to the Pipeline
InitialCorrected

23. Confidential | Copyright © Agile Testing Alliance Global Gathering
Conclusion
• Continuous DXXX…= QUALITY OF TEST CODE
• Good Test suite organization can greatly help
• Focus on automation architecture and pipeline
design with quality built-in
• Test Code will live as long as product live
• Invest on Test Code. Consider it as Safety Net.
• Remember TIER

24. Confidential | Copyright © Agile Testing Alliance Global Gathering
References
• Agile Testing- Lisa Crispin
• Pragmatic Software Testing- Rex Black
• www.thinkinginagile.com
• www. Countinuousdelivery.com
• Countinous Delivery- Jez Humble
• The Art of Agile Development-James Shore

25. Confidential | Copyright © Agile Testing Alliance Global Gathering
Reach me @
anishcheriyan@huawei.com
anishcheriyan@gmail.com
@anishcheriyan

26. Confidential | Copyright © Agile Testing Alliance Global Gathering
Thank You!
atagg@agiletestingalliance.org
@AgileTA
https://www.facebook.com/AgileTestingAlliance
contact@unicomlearning.com
@UNICOMLearning
https://www.facebook.com/joinunicom