Projects fail because they don’t test. Some fail because they test the wrong things. Others fail because they test too much. In this session, an enterprise consultant turned startup entrepreneur will share project case studies in testing atrocities and what can be learned from them. You’ll come away questioning your own testing. Check your dogma and let’s build better software. Check out TrackJS JavaScript Error Monitoring https://trackjs.com/javascript?utm_source=slides&utm_medium=slideshare&utm_term=testing

1. TERRIBLE
TESTING
case studies in

2. are you
testing
the right things?

3. I have
no idea

4. PROJECTS
SOFTWARE TESTING
and what they taught me about
5

5. toddhgardner
Todd H Gardner
!

6. 1Big Todd’s
Awesome CRM

7. 1
Delivera
CustomerRelationshipManagement
systemto replacead-hoc
processesfor internalsalesteams

9. I have no idea
what I’m doing

10. Consulting!

11. They have no idea
what they’re doing

12. 70% Code
Coverage
REQUIRED

13. Assertion-Free Testing
public void Test_MyObj_For_Coverage() {
MyObj obj = new MyObj();
try { obj.method1(); } catch (Exception e) {}
try { obj.method2(); } catch (Exception e) {}
try { obj.method3(); } catch (Exception e) {}
// 100% Coverage, W00t!
}

14. Cool Tests Bro

17. 1Big Todd’s
Awesome CRM
The Aftermath

19. They all had their rules…
1. Thou shall test everything
2. Thou shalt not ship code
3. Thou shall have more sys
4. Thou shall abstract test f
5. Thou shalt not duplicate

20. goodness
testiness
coverage
test code ratio

21. It’s unprofessional to
ship untested code
“

22. 2Big Corp’s
Video Academy

23. 2
Createa online
Video Training Platform
for crypto-zoologists

24. unit code coverage 100%
1000

25. MOCKS

26. code analysis
metrics

27. BUGS!

29. 2Big Corp’s
Video Academy
The Aftermath

30. code coverage %
1000
what do we learn?

31. code coverage %
1000
what do we learn?
not testing

32. code coverage %
1000
what do we learn?
metrics-driven
incentives

33. code coverage %
1000
what do we learn?
nothing at all

34. Testing the Mocks
public void Test_MyObj_With_Mocks() {
Mock<MyObj> mockObj1 = new Mock<MyObj>();
Mock<OtherObj> mockObj2 = new Mock<OtherObj>();
MyObj obj = new MyObj(mockObj1, mockObj2);
mockObj2.expect().toBeCalledWith(mockObj1).andReturn(mockObj1);
obj.method();
mockObj1.assertExpectations();
mockObj2.assertExpectations();
// What are we testing again?
}

35. Unit Tests

36. Test-Induced
http://david.heinemeierhansson.com/2014/test-induced-design-damage.html
Design Damage

37. 3Acme’s
Collaboration
ComputingPlatform

38. 3
Developa platformforSpiesto
Upload ConfidentialData
andCollaborateon missions.

39. Market and Technology
FRONTIERS

40. $2M / Mo
Over 200 of the
Cleverest
Designers,
Developers,
and Testers
Burning

41. Advanced
TECHNOLOGY

42. System Tests
Integration Tests
Unit Tests
Selenium UI and Services
Inter-Module Contract Tests
TDD, Conversation-based tests
Testing Pyramid

43. WE BUILT IT

44. NO ONE CAME

46. 3Acme’s
Collaboration
ComputingPlatform
The Aftermath

48. System Tests
Integration Tests
Unit Tests
Testing Pyramid

49. System Tests
Integration Tests
Unit Tests
Testing Pyramid
Interface Risk
Orchestration Risk
Functional Risk

50. System Tests
Integration Tests
Unit Tests
Testing Pyramid
Interface Risk
Orchestration Risk
Functional Risk
User Tests
Market Risk

51. Interface Risk
Orchestration Risk
Functional Risk
Market Risk
Testing Scales

52. Interface Risk
Orchestration Risk
Functional Risk
Market Risk
Testing Scales
System Tests
Integration Tests
User Tests
Unit Tests

53. 4Widget eCommerce
Monolith’s
Sell-a-majig

54. 4
Expandthe
Widget Ecommerce
platformwithmoreproducts

55. yet another
ECOMMERCE
project

57. lots of captive
USERS

58. Interface Risk
Orchestration Risk
Functional Risk
Market Risk
Testing Scales

59. Interface
Orchestration
Functional
Market
Testing Scales
System Tests
Integration Tests
User Tests
Unit Tests

60. it shipped
UGLY

61. customers kept
BUYING

62. Prints Money
the dirty old duct-taped machine that
tm

63. 4Widget eCommerce
Monolith’s
Sell-a-majig
The Aftermath

65. Monolith ECommerce Acme Collaboration
Information Technology
Uninteresting
Fragile Technology
Poorly Tested
Success!
Engineering
Advancing
Latest Technology
Well Tested
Failure!

66. Monolith ECommerce Acme Collaboration
Success! Failure!
High RiskLow Risk

67. SYSTEM
Purpose
Criticality
Lifespan
Frequency
CONFIDENCE

68. 5
My Startup

69. 5findandfix JavaScriptbugs
in modernweb applications
…andbuilda betterweb

70. {Track:js}
http://trackjs.com
JavaScript Error Monitoring

71. User Activity
AJAX Requests
Console Logging
Browser Context
Inline Source

72. Interface Risk
Orchestration Risk
Functional Risk
Market Risk
Testing Scales

73. My Money!
This time it’s

74. Launch Fast
and measure

75. Simple
Decoupled
Obvious
Abstract
&
&

76. Plumbing Services

77. Interface Risk
Orchestration Risk
Functional Risk
Market Risk
Testing Scales
System Tests
Integration Tests
User Tests
Unit Tests

78. Service
Model
Test Fixture
Other
Services
Wrapper Data
Unit
Test
Integration
Test
Testing Structure

79. “beta”
Service
Dragons be
here!
Testing Structure
Canary
Users
Analytics
Monitoring

80. FAST TO FIXis almost as good as
NEVER BROKEN
(and sometimes better)

81. PROJECTS
SOFTWARE TESTING
and what they taught me about
5

82. 1
Big Todd’s
Awesome CRM
METRICS
don’t test to hit

83. 2
Big Corp’s
Video Academy
BREAKING
test what is

84. 3
Acme’s
Collaboration
ComputingPlatform
SCARIEST
test the
thing first

85. 4
Widget eCommerce
Monolith’s
Sell-a-majig
CUSTOMER USE
tests can’tsubstitutefor

86. 5
JavaScriptError Tracking
TrackJS’s
MONITORING
can substitutefor tests

87. {Track:js}
http://trackjs.com
JavaScript Error Monitoring

88. TERRIBLE
TESTING
case studies in
toddhgardner!
" todd@trackjs.com