![2axelerant.com@shwetasharma84
About Me
{
"name": "Shweta",
"country": "India",
"city": "Pune",
"work": "Director of Quality Assurance Services",
"email": "shwetasharma@axelerant.com",
"social_media": {
"twitter": "@shwetasharma84",
"linkedin": "@shwetaneelsharma",
"github": "@shwetaneelsharma",
"skype": "@shweta_ns"
},
"hobbies": [
"classical dancing",
"globetrotting",
"watching stand-up comedy"
]
}](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
Similar to Advanced automated visual testing - DrupalCon Global 2020
Similar to Advanced automated visual testing - DrupalCon Global 2020 (20)
Recently uploaded
Recently uploaded (20)
Advanced automated visual testing - DrupalCon Global 2020
- 1. 1axelerant.com@shwetasharma84 Advanced Automated Visual Testing — Shweta Sharma @shwetasharma84
- 2. 2axelerant.com@shwetasharma84 About Me { "name": "Shweta", "country": "India", "city": "Pune", "work": "Director of Quality Assurance Services", "email": "shwetasharma@axelerant.com", "social_media": { "twitter": "@shwetasharma84", "linkedin": "@shwetaneelsharma", "github": "@shwetaneelsharma", "skype": "@shweta_ns" }, "hobbies": [ "classical dancing", "globetrotting", "watching stand-up comedy" ] }
- 3. 3axelerant.com@shwetasharma84 Axelerant from above Fully-distributed, 6 Time Zones 10 Years Of Growth 110+ Enthusiastic, Kind, Open Professionals 150+ Partner Engagements 1,000+ Open Source Contributions @shwetasharma84
- 4. 4axelerant.com@shwetasharma84 Agenda ● What is Automated Visual testing? ● Why do we need it? ● How does it work? ● Challenges with Automated Visual testing ● How to addresses these challenges? ● Strategy to design and execute automated visual tests ● Automated visual tests in the CI/CD pipeline ● A quick comparison of available tools ● How to get started?
- 5. 5axelerant.com@shwetasharma84 What is Automated Visual testing? ● Visually validate the following: ○ User Interface ■ Content ■ Page layout ○ Browsers and devices
- 6. 6axelerant.com@shwetasharma84 Why do we need it? Human Factor Larger Device/OS Matrix Larger Release Cycles Invest time to QA new features
- 7. 7axelerant.com@shwetasharma84 How does it work? Test Runner Capture screenshots at run time Report differences Update baselineBug Compare Capture baseline images First run Subsequent runs unintended intended
- 9. 9axelerant.com@shwetasharma84 BackstopJS - viewports and scenario defined
- 10. 10axelerant.com@shwetasharma84 BackstopJS - create baseline/reference shots
- 11. 11axelerant.com@shwetasharma84 BackstopJS - baseline/reference shots created
- 12. 12axelerant.com@shwetasharma84 BackstopJS - Rerun the test
- 13. 13axelerant.com@shwetasharma84 BackstopJS - Execution result (Desktop)
- 14. 14axelerant.com@shwetasharma84 BackstopJS - Execution result (Phone)
- 15. 15axelerant.com@shwetasharma84 BackstopJS - Execution result (Tablet)
- 17. 17axelerant.com@shwetasharma84 Challenge - Dynamic content (slideshow)
- 18. 18axelerant.com@shwetasharma84 Challenge - Dynamic content demo
- 19. 19axelerant.com@shwetasharma84 How few tools handle some challenges Hide ● Hide all elements queried by WebdriverIO selector strategies ● Element’s space is filled and considered in the layout of the screenshot captured ● Good candidates - Ad blocks, Images that change Remove ● Remove all elements queried by WebdriverIO selector strategies ● Element is removed from the DOM - affects the design of the page ● Good candidates - Sticky headers/footers, popover help/chat windows NOTE: Visual regression service is available till version 4.*. It is replaced by Applitools service from 5.*
- 20. 20axelerant.com@shwetasharma84 How few tools handle some challenges
- 21. 21axelerant.com@shwetasharma84 How few tools handle some challenges
- 22. 22axelerant.com@shwetasharma84 How can be Antialiasing handled? ● Use of docker ○ Environment is the same every time ● Run visual tests on cloud ○ BrowserStack ○ SauceLabs ● Select a tool which handles it implicitly
- 25. 25axelerant.com@shwetasharma84 How we integrated in our development workflow? BUILD CI PIPELINE COMMIT CODE ACCEPTANCE TESTING VISUAL REGRESSION DATA SEEDING CD PIPELINE REVIEW STAGING PRODUCTION DB-docker
- 33. 33axelerant.com@shwetasharma84 Base build selection logic in Percy * Taken from https://docs.percy.io/docs/baseline-picking-logic
- 34. 34axelerant.com@shwetasharma84 Key results achieved ● Test data ○ Time saved to handle dynamic content scenarios because of data consistency ○ Minimal efforts for Visual tests ● Seamless integration of Automation tools involved ● Visual tests validated for the site which is a replica of the wireframe ● Less number of assertions required for the Acceptance test Automation suite
- 35. 35axelerant.com@shwetasharma84 Example - No functional assertion
- 36. 36axelerant.com@shwetasharma84 Good practices to follow ● Plan the level of visual coverage needed ● Organize Test Suites ○ Browser wise ○ Feature wise ○ Device wise ● Speed up test execution by running tests in parallel ● Identify the frequency to run visual tests
- 39. 39axelerant.com@shwetasharma84 Applitools ● No pixel to pixel comparison ● In-built support for anti-aliasing ● 40+ SDKs available free to use ○ Different programming languages ○ Test Runners ○ Web/Mobile apps ○ Quick to extend your current tests to include Visual Validation ● Handles dynamic content which is also floating/shifting ● Provides seamless collaboration ● Applitools SIDE browser extension (Chrome and Firefox) ● Integrations ○ Jira ○ HP Quality Center ○ Several CI tools ● Free Support for open source projects
- 40. 40axelerant.com@shwetasharma84 Match Levels in Applitools Strict ● Recommended ● Content ● Font ● Layout ● Color ● Position of elements Layout ● Tests the layout ● Validates the alignment and relative position of all elements on the page, such as: buttons, menus, text areas, paragraphs, images, and columns ● Ignores the content, colour and other style changes between the pages Content ● Similar to Strict ● Compares content but ignores colors
- 41. 41axelerant.com@shwetasharma84 Applitools Dashboard - Baseline shots
- 43. 43axelerant.com@shwetasharma84 Applitools Dashboard - Collaborative
- 44. 44axelerant.com@shwetasharma84 Applitools Dashboard - Dynamic content
- 45. 45axelerant.com@shwetasharma84 Applitools Dashboard - Match levels
- 46. 46axelerant.com@shwetasharma84 When to use what? Static content ● Wraith ● PhantomCSS ● Spectre Shifting Content ● Applitools (SDKs and Selenium IDE extension) Dynamic content ● WebdriverIO Visual regression service ● Percy ● Shoov ● BackstopJS ● WebdriverCSS ● Gemini
- 47. 47axelerant.com@shwetasharma84 Hard hitting facts ● Efforts needed to maintain baseline images ● Respect the Test Pyramid ○ Include Unit tests, Service tests, Acceptance tests and Visual tests ● Best practices in Automating Visual tests: ○ Ensure full page validation ○ Do not have too many element specific tests ■ Avoid element locators maintenance ○ General rules for automating Acceptance tests apply here too: ■ Pick up the right candidates for Automation ■ Logical division of tests ■ Make use of Seams ● Do not expect overnight success
- 48. 48axelerant.com@shwetasharma84 References ● https://github.com/garris/BackstopJS ● http://v4.webdriver.io/guide/services/visual-regression.html ● https://applitools.com/tutorials/selenium-ide.html ● https://www.axelerant.com/resources/team-blog/visual-regression-testing-u sing-wraith ● https://www.browserstack.com/automate/webdriverio ● https://percy.io/
- 50. 50axelerant.com@shwetasharma84 Q & A ● Concept of Automated Visual testing ● Algorithm around all tools ● Challenges in Automated Visual testing ● Challenges handled by WebdriverIO and Applitools ● Gold Master Testing ● Good practices Twitter - @shwetasharma84 LinkedIn - @shwetaneelsharma GitHub - @shwetaneelsharma Email - shwetasharma@axelerant.com Skype - @shweta_ns