1. Achieving 100% Test Coverage for
Your Mobile Footprint – Issues, Options and
Strategies
An InfoStretch and Perfecto Mobile Webinar
November 12th, 2013

2. Agenda
Panelists Introduction
Mobile Testing Challenges
How Can You Optimize Quality for the Mobile App Lifecycle
Planning
Definition
Execution
Automation
How Does Perfecto’s Mobile Cloud Solves these challenges
Q&A

3. Introduction to Panelists
Manish Mathuria
Eran Kinsbruner
Founder and CTO at InfoStretch Corporation
Director of Product Marketing at Perfecto Mobile
mmathuria@infostretch.com
Erank@perfectomobile.com

4. InfoStretch - The Mobile Application Lifecycle Enablers
Matured Technology to Support Lifecycle Needs
Mobile
Applications
Requirements
Sustained User
Creative Design
Experience
and Development
Performance and
Scalability
MobiXpert
Optimize Quality
Attesto
Elemento
Qmetry
Synchro

5. Mobile Services – Custom Services for Custom Needs at Your Fingertips
Mobile Strategy
Alignment of Mobility Goals with
Business Strategy
Experience with a Vast Range
of Operating Systems and Form
Factors
SLA Driven Delivery with 30%
Reduction in Time to Market
Powered by:
Mobile Development
Specialized mobile solutions in
retail, travel, digital
entertainment and hospitality
Reusable tools, frameworks
and technology platform for
easier maintenance
Developed and deployed over
300 apps in the app stores
worldwide
Mobile QA
Mobile Sustenance
Private Cloud with 3000+
Devices
30 Day SLA for New Device
and OS Upgrades
200,000+ Apps Tested
with1500+ Device Test Cases
Real Time App Store Review
Tracking, Remediation and
Verification
Readymade Test Packs in
Retail, Banking, Travel,
Hospitality and Entertainment
Proactive Identification of User
Experience Related Issues with
Intelligent Heuristics

6. About Perfecto Mobile
6

7. Mobile Testing is NOT Easy
More Apps
Shorter release Iterations
New devices
Growing Team

8. Android Fragmentation
and Portability
11,868 distinct
Android devices,
up from 3,997
Android devices
last year

9. Delivering Quality in a Complex World is Hard
OS
Style
Screen
Version
Device
http://www.youtube.com/watch?v=X0aciNBunnQ
9

10. Mobile Quality IS a critical Challenge…
Mobile is a Market Platform

11. Plans to Scale
Success!
That won’t
happen again …
How do we
manage this?
V 1.0
Let’s get it out
the door …
KISS
Disappointment!
Read Only
Complex Txns
Quality
Forget KISS
let’s Scale
Time
V 2.0

12. Phases of Mobile Quality
Device Selection - Which OS/Device Combination is Ideal for My App?
Planning
Definition
Execution
Types of Tests - Which Ones Provide me High Coverage?
Processes - How Do I Manage All the Testing Processes? How Do I Handle
Automation?
Devices - How Do we Acquire Them? And Data Plans?
Remote Access - How Do I Do This Securely and Exclusively
Conditioning - How Do I Expose the Device to Potential Defects?
Automation – Is there ROI?
Automation – Which Tools
Automation Automation – Which Test Cases?
Release &
Sustenance
Post Release Support - How do I Manage the New OS’s and Devices
Identifying Defects - How do I Respond and Identify Defects to Ensure High Customer
Satisfaction?
End User Feedback and Usage – How do I Monitor this from App Stores

13. Planning – Device Selection Criteria
Objective: Maximum Coverage at Minimum Cost (&time)
Type of the App
User Personas
Geography
• Games
• Social
• Business
• Business
• Teen
• Social
• Asia Pac
• Americas etc.
App Functions
• Streaming
• Screen Res
• GPS etc.
Device
Popularity Form
Factor OS
Output:
Device/OS/Test
Matrix

14. Definition – With Test Management Tools
In-Depth Test Management Solution
CONTROL HUB
FOR QA
Integrates with complementary ALM tools, Simple and Easy to use

15. Execution - Mobile Labs
Geographies
Languages
3000 + Devices
• Americas
• Europe
• Asia Pac
• EFIGS, Arabic,
Japanese, Chinese,
Indian languages
• Data Plans
• Global Networks

16. Mobile Private Cloud Test Lab
Secured, Dedicated, Automated
Private Cloud
Client A
Private Cloud
Client B
Device Cradle





Device Docking Module
Remote access for verification
Certification Guidelines
Audit trail for certification
Automated Tests/Tasks
Seamless provision of new devices
 Readily available and deployable lab ( Cap-Ex Savings)
 Capability to test for global coverage (Reduce Time to market)
 Access to and testing on pre-market devices (Enhances competitiveness)

17. Mobile Testing and Automation
Mobile HTML5
Based Automation
Native Platform
Automation
Technologies
Platform Independent
Mobile Automation

18. Typical Challenges
1
How to Select Test Cases and Devices for
Mobile Automation?
2
Scripting Challenges and Dealing with
Fragmentation
3
Test Execution Challenges

19. Test Case Selection
Tool and Resource Capability
• System/Peripheral Interaction
• Interaction between multiple apps/ OS-App /Multi Domain
• Location aware, peripheral, battery life, system alerts etc.
Stability/Change
• Best Practices for other GUI Automation apply
• UI components that will change less.
• Clear understood business processes
Capability/ Complexity / Priority
• Less graphically intensive , more data and business scenarios
• Medium complexity test cases - > High - > Low
• Smoke - > High - > Medium - > Low

20. Scripting Challenges and Fragmentation

21. Widgets Render Different – Same OS

22. Inherent Device/Platform Differences

23. Form Factor Challenges
Droid X2
Droid Charge
iPhone 4

24. Dealing with Fragmentation
Browser
Open
Browser
Go To
Sign On
QTP/Selenium
Make
Payment
Text Verify
Confirm
Payment
Sign Off
Make Payment Test case
Address
ATM
Search
Browser
Open
Select
Acct View
Schedule
Payment
Sync Text
Quick
Links
Nick Name
Browser
Go To
Sign On
Sign Off
ZIP ATM
Search
Make
Payment
Sync
Image
Confirm
Payment
Component Library
Framework
iOS
Devices
Android
HTC Aria
Android
HTC Droid
BB Torch
BB 8520
Device Implementations for Sign On (10+)
LG CU620
LG
VX9000

25. Develop a Strong Test Recovery
Automation Tests will break due to
•
•
•
•
•
Device characteristics
Test script errors
Automation tool instability
Device instability
Data instability
Invest in Robust Test Recovery System
• Develop agents that can restart test components on devices e.g. iwebdriver
• Enhance test case recovery, capability to restart test case.
Invest in Logging from the test cases
• Details trace of the test
• Immediately tells you the failures

26. Automation Tool Selection Criteria
Type of Application
Sharing/collaboration
based decision
Multi - Native, Hybrid,
Web based
Cloud (Public/Private), HW
Dependency (Local)
Are we using
the right
Automation
tool?
Technologhy based
decision
Integration to QC, Script
support, Emulator support,
DOM/OCR Object
Recognition (Hybrid)
Automation is a key for both
Functional and Non Functional
Testing
HW based decision
Cross Platform support,
Device variations

27. Performance & End-User Experience – Key for Success
Per Device version
Pre-release testing (iOS7
Beta)
Apple Bug Reporter
Source: LINK
Android defect database

28. Invest in Continuous Integration
What is CI?
• Automate execution of test cases
• Integrated automated reporting, and notifications
• Prominent tools – Jenkins, hudson
Why CI?
• There is no Automation until its Automated 
• Well justified effort, easy ROI, pays off in days
• Automated trending and device comparisons.
Invest in a Test Automation lab of devices
• Dedicated automation devices

29. Perfecto Mobile Continuous Integration Workflow
Submit  Build  Test …. In Minutes vs. Hours

30. Demo