SlideShare a Scribd company logo
1 of 27
Download to read offline
Testing Best Practices


         Kevin Whinnery
        Platform Evangelist
         Appcelerator Inc.
         @kevinwhinnery

    kwhinnery@appcelerator.com
Testing Best Practices


                         • The Challenge of
                           Testing Rich Client
                           Applications
                         • Levels of Testing
                         • Titanium Testing Tools
                         • The Future of Testing
                           Titanium Apps
                         • Q&A
The Challenge of Rich
Client Testing
Rich Client Testing Challenges


A Few Challenges:     Testing rich client applications
•   Relevant tests?   remains a challenge today due to
•   Tooling?          the visual and human aspects of
•   Automation?
•   Production        client applications.
    environment?
•   Human
    interaction?
How relevant are our tests?


                   • Model tier testing is highly
                     doable, and test friendly
                   • UI testing is hard to do in a
                     meaningful way:
                      • Does code work in a long-
                         running scenario?
                      • If I programmatically build a
                         UI structure, how close is that
                         to the real UI structure at
                         runtime?
                      • I can test object
                         properties, but reliably
                         ensuring visual integrity is
What tools do we use?


                  • In JS, client side test tools are
                    somewhat limited in functionality
                    and scope
                  • What is available is more geared
                    toward simple applications
                     • …and typically tied to the
                        browser, in the case of
                        JavaScript
How do we automate tests?


                 • Scripting user interaction is
                   difficult, especially in native
                   mobile apps
                 • Requires multi-platform
                   environment
                 • Requires some sort of
                   connection to a running
                   simulator or device, which must
                   also be scripted
How close are we to prod conditions?


                  • A simulator only goes so far
                  • Need to test on a variety of
                    devices
                  • Mobile is a hostile and volatile
                    environment:
                     • Data connectivity?
                     • Network speeds?
                     • Software/hardware
                        combinations (iOS has this
                        pain now too)
                     • Background processes?
                     • Geographic locations?
How is our “look and feel”?


                   • Does our app “look right” with
                     different data sets?
                   • Are our buttons responsive
                     enough, or do loading screens
                     take too long?
                   • Are there operations which are
                     unintuitive, or error prone?
                   • Are the properties of the UI we
                     are changing and animating
                     actually reflected in the UI how
                     we expect?
These problems can
all be overcome, but
testing can become a
classic struggle
between…
Time
Quality
And Resources
Diminishing Returns


Consider:               Test automation can become
•   How long will I     extremely expensive. Frameworks
    support this        need to be maintained, infrastructure
    app?
•   How useful is the   needs to be created, and time needs
    test automation I
    am adding – how
                        to be spent writing, scripting, or
    true to life are    recording tests.
    my test results?
•   Automation
    versus human        Make sure the investment is
    testing
                        justified.
Levels of Testing
Levels of Testing


Types of Testing:        Depending on the scope, shelf
•   Ad hoc               life, and team size for every
•   Automated Unit       application, only certain levels of
    Testing
•   Continuous           testing may be needed.
    Integration
•   Automated
    Functional Testing   You probably already know these by
•   Crowdsourcing
                         some name, but let’s establish a
                         common vocabulary.
Levels of Testing


Types of Testing:        Ad hoc testing by a single tester or
•   Ad hoc               group of testers may be sufficient for
•   Automated Unit       small applications with a short shelf
    Testing
•   Continuous           life.
    Integration
•   Automated
    Functional Testing
•   Crowdsourcing
Levels of Testing


Types of Testing:        Programmatic unit testing can
•   Ad hoc               provide a baseline comfort level with
•   Automated Unit       code in a project, giving the
    Testing
•   Continuous           developer knowledge that public
    Integration
•   Automated
                         APIs have not regressed and are
    Functional Testing   working as expected.
•   Crowdsourcing
Levels of Testing


Types of Testing:        Continuous integration can make
•   Ad hoc               use of an existing suite of automated
•   Automated Unit       unit tests, and ensure that all test
    Testing
•   Continuous           suites are passing as new code is
    Integration
•   Automated
                         committed to the project.
    Functional Testing
•   Crowdsourcing
                         Continuous integration becomes
                         especially useful as a team
                         increases in size and activity.
Levels of Testing


Types of Testing:        Automated functional testing tests
•   Ad hoc               actual application interactions, and
•   Automated Unit       inspects the state of an application
    Testing
•   Continuous           at known points.
    Integration
•   Automated
    Functional Testing   This type of testing is sometimes
•   Crowdsourcing
                         referred to as “record and play”
                         testing.
Levels of Testing


Types of Testing:        Crowdsourcing deploys an
•   Ad hoc               application to real humans around
•   Automated Unit       the world, who can test applications
    Testing
•   Continuous           on different data
    Integration
•   Automated
                         networks, devices, and locations to
    Functional Testing   provide real-world feedback on
•   Crowdsourcing
                         application functionality.

                         Can also be employed to provide
                         feedback on look and feel and other
                         human elements.
Where We Are Today


Types of Testing:        Appcelerator (and our community)
•   Ad hoc               have developed test automation
•   Automated Unit       tools to provide a baseline level of
    Testing
•   Continuous           comfort with application functionality.
    Integration
•   Automated
    Functional Testing   The testability of your application
•   Crowdsourcing
                         can be greatly enhanced by
                         employing modular design.
Demo: Testable
Titanium apps with
behave.js
Where We’ll Be Tomorrow


Types of Testing:        Appcelerator, the community, and
•   Ad hoc               our commercial partners are
•   Automated Unit       collaborating on testing solutions
    Testing
•   Continuous           across the board.
    Integration
•   Automated
    Functional Testing   Visit with SOASTA and uTest in the
•   Crowdsourcing
                         hallways!
To Sum Up


            • Test automation is a
              great tool for ensuring
              quality
            • For rich client
              apps, useful tests have
              always been a
              challenge
            • Choose the level of
              testing appropriate for
              the project
            • Join us in reducing
              barriers to testing and
              building awesome tools
To Sum Up


            Downloads/Resources:

            • http://bit.ly/
               • codestrong_src
               • ti_test_slides
Kevin Whinnery
     @kevinwhinnery

kwhinnery@appcelerator.com

More Related Content

More from Axway Appcelerator

Stop Debating, Start Measuring
Stop Debating, Start MeasuringStop Debating, Start Measuring
Stop Debating, Start MeasuringAxway Appcelerator
 
Mobile & The New Experience Economy (And What it Means for IT)
Mobile & The New Experience Economy  (And What it Means for IT)Mobile & The New Experience Economy  (And What it Means for IT)
Mobile & The New Experience Economy (And What it Means for IT)Axway Appcelerator
 
Apps, APIs & Analytics: What "Mobile First" Really Means
Apps, APIs & Analytics: What "Mobile First" Really MeansApps, APIs & Analytics: What "Mobile First" Really Means
Apps, APIs & Analytics: What "Mobile First" Really MeansAxway Appcelerator
 
Appcelerator Presentation Template
Appcelerator Presentation TemplateAppcelerator Presentation Template
Appcelerator Presentation TemplateAxway Appcelerator
 
Codestrong 2012 keynote jonathan rende, appcelerator's vp of products
Codestrong 2012 keynote   jonathan rende, appcelerator's vp of productsCodestrong 2012 keynote   jonathan rende, appcelerator's vp of products
Codestrong 2012 keynote jonathan rende, appcelerator's vp of productsAxway Appcelerator
 
Codestrong 2012 keynote jeff haynie, appcelerator's ceo
Codestrong 2012 keynote   jeff haynie, appcelerator's ceoCodestrong 2012 keynote   jeff haynie, appcelerator's ceo
Codestrong 2012 keynote jeff haynie, appcelerator's ceoAxway Appcelerator
 
Codestrong 2012 keynote how to build a top ten app
Codestrong 2012 keynote   how to build a top ten appCodestrong 2012 keynote   how to build a top ten app
Codestrong 2012 keynote how to build a top ten appAxway Appcelerator
 
Codestrong 2012 breakout session at&t api platform and trends
Codestrong 2012 breakout session  at&t api platform and trendsCodestrong 2012 breakout session  at&t api platform and trends
Codestrong 2012 breakout session at&t api platform and trendsAxway Appcelerator
 
Codestrong 2012 breakout session what's new in titanium studio
Codestrong 2012 breakout session   what's new in titanium studioCodestrong 2012 breakout session   what's new in titanium studio
Codestrong 2012 breakout session what's new in titanium studioAxway Appcelerator
 
Codestrong 2012 breakout session using appcelerator cloud services in your ...
Codestrong 2012 breakout session   using appcelerator cloud services in your ...Codestrong 2012 breakout session   using appcelerator cloud services in your ...
Codestrong 2012 breakout session using appcelerator cloud services in your ...Axway Appcelerator
 
Codestrong 2012 breakout session the role of cloud services in your next ge...
Codestrong 2012 breakout session   the role of cloud services in your next ge...Codestrong 2012 breakout session   the role of cloud services in your next ge...
Codestrong 2012 breakout session the role of cloud services in your next ge...Axway Appcelerator
 
Codestrong 2012 breakout session new device platform support for titanium
Codestrong 2012 breakout session   new device platform support for titaniumCodestrong 2012 breakout session   new device platform support for titanium
Codestrong 2012 breakout session new device platform support for titaniumAxway Appcelerator
 
Codestrong 2012 breakout session mobile platform and infrastructure
Codestrong 2012 breakout session   mobile platform and infrastructureCodestrong 2012 breakout session   mobile platform and infrastructure
Codestrong 2012 breakout session mobile platform and infrastructureAxway Appcelerator
 
Codestrong 2012 breakout session making money on appcelerator's marketplace
Codestrong 2012 breakout session   making money on appcelerator's marketplaceCodestrong 2012 breakout session   making money on appcelerator's marketplace
Codestrong 2012 breakout session making money on appcelerator's marketplaceAxway Appcelerator
 
Codestrong 2012 breakout session live multi-platform testing
Codestrong 2012 breakout session   live multi-platform testingCodestrong 2012 breakout session   live multi-platform testing
Codestrong 2012 breakout session live multi-platform testingAxway Appcelerator
 
Codestrong 2012 breakout session leveraging titanium as part of your mobile...
Codestrong 2012 breakout session   leveraging titanium as part of your mobile...Codestrong 2012 breakout session   leveraging titanium as part of your mobile...
Codestrong 2012 breakout session leveraging titanium as part of your mobile...Axway Appcelerator
 
Codestrong 2012 breakout session i os internals and best practices
Codestrong 2012 breakout session   i os internals and best practicesCodestrong 2012 breakout session   i os internals and best practices
Codestrong 2012 breakout session i os internals and best practicesAxway Appcelerator
 
Codestrong 2012 breakout session introduction to mobile web and best practices
Codestrong 2012 breakout session   introduction to mobile web and best practicesCodestrong 2012 breakout session   introduction to mobile web and best practices
Codestrong 2012 breakout session introduction to mobile web and best practicesAxway Appcelerator
 
Codestrong 2012 breakout session how to win bigger mobile deals
Codestrong 2012 breakout session   how to win bigger mobile dealsCodestrong 2012 breakout session   how to win bigger mobile deals
Codestrong 2012 breakout session how to win bigger mobile dealsAxway Appcelerator
 
Codestrong 2012 breakout session how to develop your own modules
Codestrong 2012 breakout session   how to develop your own modulesCodestrong 2012 breakout session   how to develop your own modules
Codestrong 2012 breakout session how to develop your own modulesAxway Appcelerator
 

More from Axway Appcelerator (20)

Stop Debating, Start Measuring
Stop Debating, Start MeasuringStop Debating, Start Measuring
Stop Debating, Start Measuring
 
Mobile & The New Experience Economy (And What it Means for IT)
Mobile & The New Experience Economy  (And What it Means for IT)Mobile & The New Experience Economy  (And What it Means for IT)
Mobile & The New Experience Economy (And What it Means for IT)
 
Apps, APIs & Analytics: What "Mobile First" Really Means
Apps, APIs & Analytics: What "Mobile First" Really MeansApps, APIs & Analytics: What "Mobile First" Really Means
Apps, APIs & Analytics: What "Mobile First" Really Means
 
Appcelerator Presentation Template
Appcelerator Presentation TemplateAppcelerator Presentation Template
Appcelerator Presentation Template
 
Codestrong 2012 keynote jonathan rende, appcelerator's vp of products
Codestrong 2012 keynote   jonathan rende, appcelerator's vp of productsCodestrong 2012 keynote   jonathan rende, appcelerator's vp of products
Codestrong 2012 keynote jonathan rende, appcelerator's vp of products
 
Codestrong 2012 keynote jeff haynie, appcelerator's ceo
Codestrong 2012 keynote   jeff haynie, appcelerator's ceoCodestrong 2012 keynote   jeff haynie, appcelerator's ceo
Codestrong 2012 keynote jeff haynie, appcelerator's ceo
 
Codestrong 2012 keynote how to build a top ten app
Codestrong 2012 keynote   how to build a top ten appCodestrong 2012 keynote   how to build a top ten app
Codestrong 2012 keynote how to build a top ten app
 
Codestrong 2012 breakout session at&t api platform and trends
Codestrong 2012 breakout session  at&t api platform and trendsCodestrong 2012 breakout session  at&t api platform and trends
Codestrong 2012 breakout session at&t api platform and trends
 
Codestrong 2012 breakout session what's new in titanium studio
Codestrong 2012 breakout session   what's new in titanium studioCodestrong 2012 breakout session   what's new in titanium studio
Codestrong 2012 breakout session what's new in titanium studio
 
Codestrong 2012 breakout session using appcelerator cloud services in your ...
Codestrong 2012 breakout session   using appcelerator cloud services in your ...Codestrong 2012 breakout session   using appcelerator cloud services in your ...
Codestrong 2012 breakout session using appcelerator cloud services in your ...
 
Codestrong 2012 breakout session the role of cloud services in your next ge...
Codestrong 2012 breakout session   the role of cloud services in your next ge...Codestrong 2012 breakout session   the role of cloud services in your next ge...
Codestrong 2012 breakout session the role of cloud services in your next ge...
 
Codestrong 2012 breakout session new device platform support for titanium
Codestrong 2012 breakout session   new device platform support for titaniumCodestrong 2012 breakout session   new device platform support for titanium
Codestrong 2012 breakout session new device platform support for titanium
 
Codestrong 2012 breakout session mobile platform and infrastructure
Codestrong 2012 breakout session   mobile platform and infrastructureCodestrong 2012 breakout session   mobile platform and infrastructure
Codestrong 2012 breakout session mobile platform and infrastructure
 
Codestrong 2012 breakout session making money on appcelerator's marketplace
Codestrong 2012 breakout session   making money on appcelerator's marketplaceCodestrong 2012 breakout session   making money on appcelerator's marketplace
Codestrong 2012 breakout session making money on appcelerator's marketplace
 
Codestrong 2012 breakout session live multi-platform testing
Codestrong 2012 breakout session   live multi-platform testingCodestrong 2012 breakout session   live multi-platform testing
Codestrong 2012 breakout session live multi-platform testing
 
Codestrong 2012 breakout session leveraging titanium as part of your mobile...
Codestrong 2012 breakout session   leveraging titanium as part of your mobile...Codestrong 2012 breakout session   leveraging titanium as part of your mobile...
Codestrong 2012 breakout session leveraging titanium as part of your mobile...
 
Codestrong 2012 breakout session i os internals and best practices
Codestrong 2012 breakout session   i os internals and best practicesCodestrong 2012 breakout session   i os internals and best practices
Codestrong 2012 breakout session i os internals and best practices
 
Codestrong 2012 breakout session introduction to mobile web and best practices
Codestrong 2012 breakout session   introduction to mobile web and best practicesCodestrong 2012 breakout session   introduction to mobile web and best practices
Codestrong 2012 breakout session introduction to mobile web and best practices
 
Codestrong 2012 breakout session how to win bigger mobile deals
Codestrong 2012 breakout session   how to win bigger mobile dealsCodestrong 2012 breakout session   how to win bigger mobile deals
Codestrong 2012 breakout session how to win bigger mobile deals
 
Codestrong 2012 breakout session how to develop your own modules
Codestrong 2012 breakout session   how to develop your own modulesCodestrong 2012 breakout session   how to develop your own modules
Codestrong 2012 breakout session how to develop your own modules
 

Codestrong 2012 breakout session testing best practices unit and functional automation

  • 1. Testing Best Practices Kevin Whinnery Platform Evangelist Appcelerator Inc. @kevinwhinnery kwhinnery@appcelerator.com
  • 2. Testing Best Practices • The Challenge of Testing Rich Client Applications • Levels of Testing • Titanium Testing Tools • The Future of Testing Titanium Apps • Q&A
  • 3. The Challenge of Rich Client Testing
  • 4. Rich Client Testing Challenges A Few Challenges: Testing rich client applications • Relevant tests? remains a challenge today due to • Tooling? the visual and human aspects of • Automation? • Production client applications. environment? • Human interaction?
  • 5. How relevant are our tests? • Model tier testing is highly doable, and test friendly • UI testing is hard to do in a meaningful way: • Does code work in a long- running scenario? • If I programmatically build a UI structure, how close is that to the real UI structure at runtime? • I can test object properties, but reliably ensuring visual integrity is
  • 6. What tools do we use? • In JS, client side test tools are somewhat limited in functionality and scope • What is available is more geared toward simple applications • …and typically tied to the browser, in the case of JavaScript
  • 7. How do we automate tests? • Scripting user interaction is difficult, especially in native mobile apps • Requires multi-platform environment • Requires some sort of connection to a running simulator or device, which must also be scripted
  • 8. How close are we to prod conditions? • A simulator only goes so far • Need to test on a variety of devices • Mobile is a hostile and volatile environment: • Data connectivity? • Network speeds? • Software/hardware combinations (iOS has this pain now too) • Background processes? • Geographic locations?
  • 9. How is our “look and feel”? • Does our app “look right” with different data sets? • Are our buttons responsive enough, or do loading screens take too long? • Are there operations which are unintuitive, or error prone? • Are the properties of the UI we are changing and animating actually reflected in the UI how we expect?
  • 10. These problems can all be overcome, but testing can become a classic struggle between…
  • 11. Time
  • 14. Diminishing Returns Consider: Test automation can become • How long will I extremely expensive. Frameworks support this need to be maintained, infrastructure app? • How useful is the needs to be created, and time needs test automation I am adding – how to be spent writing, scripting, or true to life are recording tests. my test results? • Automation versus human Make sure the investment is testing justified.
  • 16. Levels of Testing Types of Testing: Depending on the scope, shelf • Ad hoc life, and team size for every • Automated Unit application, only certain levels of Testing • Continuous testing may be needed. Integration • Automated Functional Testing You probably already know these by • Crowdsourcing some name, but let’s establish a common vocabulary.
  • 17. Levels of Testing Types of Testing: Ad hoc testing by a single tester or • Ad hoc group of testers may be sufficient for • Automated Unit small applications with a short shelf Testing • Continuous life. Integration • Automated Functional Testing • Crowdsourcing
  • 18. Levels of Testing Types of Testing: Programmatic unit testing can • Ad hoc provide a baseline comfort level with • Automated Unit code in a project, giving the Testing • Continuous developer knowledge that public Integration • Automated APIs have not regressed and are Functional Testing working as expected. • Crowdsourcing
  • 19. Levels of Testing Types of Testing: Continuous integration can make • Ad hoc use of an existing suite of automated • Automated Unit unit tests, and ensure that all test Testing • Continuous suites are passing as new code is Integration • Automated committed to the project. Functional Testing • Crowdsourcing Continuous integration becomes especially useful as a team increases in size and activity.
  • 20. Levels of Testing Types of Testing: Automated functional testing tests • Ad hoc actual application interactions, and • Automated Unit inspects the state of an application Testing • Continuous at known points. Integration • Automated Functional Testing This type of testing is sometimes • Crowdsourcing referred to as “record and play” testing.
  • 21. Levels of Testing Types of Testing: Crowdsourcing deploys an • Ad hoc application to real humans around • Automated Unit the world, who can test applications Testing • Continuous on different data Integration • Automated networks, devices, and locations to Functional Testing provide real-world feedback on • Crowdsourcing application functionality. Can also be employed to provide feedback on look and feel and other human elements.
  • 22. Where We Are Today Types of Testing: Appcelerator (and our community) • Ad hoc have developed test automation • Automated Unit tools to provide a baseline level of Testing • Continuous comfort with application functionality. Integration • Automated Functional Testing The testability of your application • Crowdsourcing can be greatly enhanced by employing modular design.
  • 24. Where We’ll Be Tomorrow Types of Testing: Appcelerator, the community, and • Ad hoc our commercial partners are • Automated Unit collaborating on testing solutions Testing • Continuous across the board. Integration • Automated Functional Testing Visit with SOASTA and uTest in the • Crowdsourcing hallways!
  • 25. To Sum Up • Test automation is a great tool for ensuring quality • For rich client apps, useful tests have always been a challenge • Choose the level of testing appropriate for the project • Join us in reducing barriers to testing and building awesome tools
  • 26. To Sum Up Downloads/Resources: • http://bit.ly/ • codestrong_src • ti_test_slides
  • 27. Kevin Whinnery @kevinwhinnery kwhinnery@appcelerator.com