Thomas Gronbach, Tobias Dreyschultze Keynote Veranstaltung: M-Days 2013 Blog by Messe Frankfurt for the Digital Business: http://connected.messefrankfurt.com/en/

1. The Enterprise Goes Mobile
Thomas Gronbach
Tobias Dreyschultze
Keynote
February 6th 2013

2. February 8, 2013
About Keynote
• Founded May 1995
• Revenue: FY2012 – 124M / Q1FY2013 – 34M
• Public, NASDAQ: “KEYN” since Sept, 1999
• 450+ employees
• 4,000 customers
Keynotes Leadership
• On-demand infrastructure of over 7,000
measurement computers and mobile devices
in more than 275 locations around the world.
• DeviceAnywhere: #1 Mobile Testing Platform
• Named by Forbes as ‘One of the Best 100
Companies in America’ with Under $1 Billion
in Sales.
© 2013 Keynote Systems 2

3. February 8, 2013
Agenda
Mobile is a TOP Priority for Enterprises
Who are Enterprises?
What does it Mean TOP Priority?
What does it Mean for Mobile Development?
Effect of TOP Priority on Mobile Development
The (formerly Web) and now Mobile Developer
Mobile Development Puzzle
Enterprises adopting Mobile Testing
© 2013 Keynote Systems 3

4. February 8, 2013
Who chooses this as a TOP Priority? Who is an Enterprise?
Some Mobile Enterprise Profiles
Type of Company Consumer Applications Internal Applications
Main website (HTML) Internal corporate website (HTML)
Financial Institution Mobile-optimized website (HTML) Mobile-optimized corporate website
Tablet-optimized website (HTML) (HTML)
Commercial banking app Trading app (Native: BB)
(Native: iOS, Android, BB) Expense app (HTML)
Investment app
(Native: iOS, Android, BB)
Mobile payment app
(Native: iOS, Android)
Retailer Main website (HTML) POS app (Native: iOS)
Mobile-optimized website (HTML) Time tracking (HTML)
Mobile app (Hybrid – native with some Inventory tracking (HTML)
web content: iOS, Android)
© 2013 Keynote Systems

5. February 8, 2013
What does it mean TOP Priority?
Enterprises Investments
Secure Mobile Apps
Qualitatively Sound Mobile Apps
Reliable Mobile Apps
Available Mobile Apps
Mitigate Mobile Risk
© 2013 Keynote Systems 5

6. February 8, 2013
What does it mean for mobile development?
Get S, Q, R, A, M into your work
Good quality development
Test before production
Test well before production
Test a lot before production
Test during production
Test before development
Don’t forget that testing gets S,Q,R,A,M
into your work
© 2013 Keynote Systems 6

7. February 8, 2013
The Mobile Developer
Developers are facilitator for Enterprise’s TOP Priority
Thought leader
Differentiated developer do things first
Real world testing
Real device testing
Simple, easy access for everyone
© 2013 Keynote Systems 7

8. February 8, 2013
Mobile Development Puzzle
Testing in a highly heterogeneous and diverse environment
App type: native, web, hybrid
OS variety QA Effort Portion of Development
Timeline
Device variety
Differing display sizes
Effort (Total Hours Spent)
Differences in processing speed,
memory
Android customizations
Device life cycle – 12 to 18 months vs.
2 – 3 years
Connectivity
Traditional Products Mobile Products
Network latency
Requirements/Design Development QA
Connectivity fluctuation
© 2013 Keynote Systems
8

9. February 8, 2013
Keynote’s DeviceAnywhere Platform
© 2013 Keynote Systems

10. February 8, 2013
Market Leader in Devices Supported
DeviceAnywhere platform has over 1,000 devices* on all OS’ including…
• Android
• iOS
• Windows Mobile
• BlackBerry
Just announced…
• Support of iPhone 5
• – available now!
• Windows 8 – coming soon!
• BlackBerry 10 – coming soon!
*Number of devices available includes duplicate models.
© 2013 Keynote Systems 10

11. February 8, 2013
How it Works - Keynote’s DeviceAnywhere Mobile Testing Datacenter
© 2013 Keynote Systems 11

12. February 8, 2013
How it Works
Live Mobile Device
Rack mountable
tabs
Power Switch Status/Service
Indicators Hardwiring Harness Device Standard I/O
RF Antenna
Data Cable Server Connection SIM Switcher (USB)
Power SIM Card Slot
Connection (USB) (USB)
© 2013 Keynote Systems

13. February 8, 2013
DeviceAnywhere Free
Changing the Mobile Testing landscape
• Lets users spot check mobile
websites on real, remote
smartphones.
• Eliminates the cost of device
ownership, management, and
data plans.
• Introduces development and QA
teams to capabilities of cloud-based
testing solutions.
• No cost. No commitment.
© 2013 Keynote Systems 13

14. February 8, 2013
Large US Bank
Three Years Experience in Mobile Testing
Consumer Banking division’s QA team initially signed up for annual
subscription on shared testing environment
Later added other divisions and replaced cloud service with a licensed
dedicated enterprise system
Also added test automation for enterprise style testing of multiple test
cases integrated with their QA tools
Strategy powered by having a portfolio of testing products – delivering
in both cloud and systems configurations.
© 2013 Keynote Systems 14

15. February 8, 2013
Humana: Improving the Quality of their Mobile Apps & Websites
Humana
Fortune-100 healthcare company with over 40,000 employees
Healthcare operations in all 50 US states
Goal
Provides consumers and businesses with mobile access anywhere
any channel across various healthcare operations
HTML5-based web apps, mobile websites and native mobile apps
Approach
Technical drivers
Support emerging mobile technologies
150 different test cases
Licensed 50 different devices/OS
Business driver
Humana corporate initiative: expand mobile access to patient information
Develop mobile process improvements to support upcoming mobile web and applications
portfolio additions
Device flexibility supporting changing market and project requirements
Provide reliable and expedient access to consumers’ health information whenever needed
© 2013 Keynote Systems 15

16. February 8, 2013
TomTom: Monitoring their
Solution
• Keynote’s DeviceAnywhere TCE Monitoring platform provides remote access to TomTom
devices, including its latest generation devices and the iPhone 4
• Connection to a live mobile network in France
• Housed in a data center in Paris, Keynote created a customized environment, providing
GPS connectivity supporting functionality and performance monitoring scenarios
Benefit
• TomTom’s operations receives real-time alerts about software, hardware and network
issues that could affect the end-user experience.
• Alerts are pre-defined and require no additional involvement from TomTom’s
development team
• Focus on developing the next-generation features and services
© 2013 Keynote Systems 16

17. February 8, 2013
Meet Keynote and get hands-on experience with DeviceAnywhere
Keynote at M-Days
Hands-on demo at booth IN15
Humana case study – request 3 pager for details
Sign up for DeviceAnywhere FREE
www.keynote.com/dafree
Free Mobile Testing Event in Munich, February 19th 2013
Hands-on Sessions
Sign up today: www.keynote.com/munich
Mobile World Congress, Feb 25-28, 2013, Barcelona
Booth 6C71
© 2013 Keynote Systems
17

18. February 8, 2013
Demo
Thank You
Thomas Gronbach
Thomas.Gronbach@keynote.com
www.keynotedeviceanywhere.com
http://mobileapplicationtestingtimes.wordpress.com/
https://twitter.com/DevAnywhere
© 2013 Keynote Systems

19. Appendix

20. Mobile is a TOP Priority
CIOs provided their feedback in a survey at Gartner’s 2012 CIO Leadership Forum, themed
Amplify the Enterprise
Mark P. McDonald, Ph.D, Group VP & Head of Research, Gartner Executive Programs – January 2012
© 2013 Keynote Systems 20

21. Elements of a Test Strategy
Testing Platform
Real device testing platform
Remote access to enable geographically diverse teams
Flexible device deployment options
Security and management
Devices Rationalization
Device
Optimize coverage (or minimize risk)
Control costs (time, equipment)
Test Automation
When and where to automate
Efficient automation of web, hybrid, and native apps
Flexible scripting options to enable QA engineers with different skillsets
© 2013 Keynote Systems 21

22. Real Device Testing Platform: Remote Access to Real Devices
Benefits
Enables management and storage of
test assets and test results
Collaboration environment for QA
What device input and output capabilities are required?
Keys; tapping and swiping touch screens
Opening and closing devices, connecting/disconnecting the battery, accelerometer support,
audio, backlight, camera
Viewing the LCD with accurate pixel-by-pixel displays
© 2013 Keynote Systems 22

23. Multiple Options for Accessing Real Devices
Shared Devices
Devices access to hundreds of smart devices
24X7
Use for compatibility testing, “untrusted
device” testing
Private Devices
24X7 access to your devices
Enables geographically diverse team
Inside corporate firewall or external
Local Devices
Plug your smart devices on hand directly into
your local computer
Make use of existing assets, great for local
teams
© 2013 Keynote Systems
Keynote Confidential 23

24. Approach to Rationalizing Test Devices
Reduce required test devices by defining key criteria (e.g. OS version, screen resolution, CPU, etc.) that will
impact the performance of your application and optimize for popular devices that provide coverage across
these criteria
Device Planner: a free tool by Keynote
Prioritize key criteria
Choose the target # of devices
The Device Planner generates a device list
based on device popularity, by finding devices
Device Planner: that provide coverage across key criteria
Key Criteria Example: The Samsung Galaxy SIII is chosen
first as the most popular Android device. The
Galaxy Nexus is another popular device, but as
its OS is also Android 4.0, it moves to another
device, e.g. the LG Spectrum on Android 2.3,
to maximize coverage across OS versions.
Device Planner:
Optimized Device The Device Planner can be found at
List tce.deviceanywhere.com
© 2013 Keynote Systems

25. Test Automation
As with software QA, test automation of mobile apps both save
money and improve the effectiveness of your QA test process
Save Money
Automate smoke tests and regression tests to reduce cost of manual testing
Find bugs earlier in the process when it is cheaper to fix them
Execute more tests in less time, improving time to market and/or
increasing test coverage
Execute 2.5 – 3X faster than manual tests
Schedule tests to run offline; utilize up to 24 hours of each day
Improve Quality
Find more bugs: increase test case coverage, reduce tester fatigue
Offload testers from repetitive tasks, allowing them to focus on product
quality (e.g. exploratory testing)
Increase device coverage
Address quality in an increasingly fragmented device market
The more models you test against, the more efficiency you gain
© 2013 Keynote Systems 25

26. February 8, 2013
Mobile Application Testing Stages: Where to Automate?
Smoke New Feature Regression Compatibility Exploratory
Testing Testing Testing Testing Testing
Deep testing of Functional testing Exploration of
Goals Basic acceptance, Compatibility
new areas of across all existing functionality from
build acceptance across devices
functionality areas of product customer viewpoint
Test Cases
Frequency
Methodology Very high value Low value to High value to Medium value to Not automated
to automate! automate automate automate
# of Devices
Optimal Test Automated Manual Testing Automated Manual or Manual Testing
Strategy Testing Testing Automated Testing
© 2013 Keynote Systems 26

27. Use Case: Automation
One customer had a series of 100 test cases run across one Android and one
iOS device that were required to verify each new build. Two QA engineers
were effectively completing 40 test cases per day, or requiring 1 week of
elapsed time (10 man-days)
Customer utilized DeviceAnywhere Test Center Enterprise Automation to
automate all 200 test cases, which can now be scheduled to automatically
run overnight when a new build is generated.
As a result, the customer improved time to market by 1 week, while
freeing up QA resources for other testing efforts.
© 2013 Keynote Systems 27

28. DeviceAnywhere Platform – Enabling Testing of Mobile Apps
Test hybrid and web applications, from a single platform, allowing you to:
• Test on real devices, ensuring test accuracy
• Enable all of your remote and local employees,
with or without existing scripting skills
• Save time by automating native, web, and
hybrid application testing
© 2013 Keynote Systems 28

29. THANK YOU!