In this presentation you will learn how Farm Credit Services of America/Frontier Farm Credit transformed their quality practices and tooling to bring visibility and consistency to Enterprise Quality, including: testing as a team approach, creating an automated test architecture, measuring progress with dashboards and standardizing on a set of testing tools.

1. Road Map to Enterprise Quality
Jeff Bramwell
VP – Enterprise Architecture
Farm Credit Services of America
Jeff.Bramwell@fcsamerica.com
@jbramwell

2. Agenda
• Enterprise Quality Vision
• Strategy
• Tooling

3. Enterprise Quality Vision
• Why?

4. Why Enterprise Quality?
125+
Applications
& Services
950+
deployments
in 2016
11 AppDev
teams
1,000+
deployments
in 2017

5. Why Enterprise Quality?

6. Strategy
• Shift Left
• People
• Process
• Tools

7. Shifting Left

8. Success Takes People!
• Quality Assurance role – not just for testing any more, also…
• System health analysis
• Visibility & understanding
• Peer group
• Collaboration
• Development
• Ability to understand and write code

9. Focus on Enterprise Quality
• Enterprise architect with applied focus on enterprise quality
• …and health
• …and visibility

10. QA Role
QAEQAD

11. Lean Quality Delivery (LQD)
• Vision
• To be the chosen partner
• Goals
• Functional quality
• Structural quality
• Process quality
• Strategies
• Consistent across teams
• Tactics
• Can vary by team and project
• Teams revisit LQD/tactics at the end/beginning of projects

12. Lean Quality Delivery (LQD) – Cont’d
Lean Quality Delivery (LQD)
Vision Goal Strategy Tactics
Score
(1-5)
ToBetheChosenPartner
FunctionalQuality
1. Delivers the desired value
 Story writing and review
 UI automation testing
 Performance testing (Does it meet SLA needs?)
 Service-level testing
 Unit testing
 Discussion with development teams
2. Deliberate testing plan at the
story and project level
 Testing starts with the creation of the story and review
 Testing is a first-class citizen
 Tests should answer questions
o What makes the story done?
o How is the user going to use it?
3. High reliability, minimal defects
 Pairing/Code reviews
 Health monitoring (Are we notified when something goes awry?)
4. Delightful user experience
 User interviews (in our case, this is typically developers)
 Process mapping
 Prototyping
 Follow-up discussions/surveys
StructuralQuality
1. Design for testability
 Incorporate design patterns (as appropriate)
 TDD/Code reviews
 Continuous Integration
 Continuous Delivery
2. Security best practices
 Security training for developers
 Static and dynamic code scanning (for public-/partner-facing
apps)
3. Follows project guidance
 Application reviews
 Reuse of enterprise components
 Reference Tech Radar (and follow the process for new
technologies)
 Create Project Start Architecture document
ProcessQuality
1. Responsive support
 Proactive support (e.g. Do we send “ack” within 15 minutes?)
 Someone takes ownership of issue and “walks it through”
2. Consistent Process
 The team can describe the process and how it impacts quality
 The team implements the agreed upon tactics
 Reviewing and grading of the team this document
3. Predictable Delivery
 Measure Cycle Time
 Groom Backlog
 Manage Budget
 Manage Timeline
 Automated deployments
4. Visibility & Transparency of
 Count defects and kickbacks
 Count production-related issues
 System Health (for systems under our team’s control)
5. Routinely follows process for
feedback and learning
 Retrospectives
 Demos
 5 Why's for production defects

13. Lean Quality Delivery (LQD) – Cont’d

14. Process – CI/CD

15. CD vs. CD
• Automation is key
• But which is it – Continuous Delivery or Continuous Deployment?See the
difference?

16. CD vs. CD – Cont’d
• What’s the difference?
• Trust!
• It will take time

17. Tooling

18. Version Control (VSTS)
• Use version control as
the single source of
truth for Automation
Scripts.
Dev
• VSTS
• Visual Studio
• GIT
• Common lib
• Project Repo
QA
• VSTS
• Visual Studio
• GIT (SourceTree)
• QA
• QA Repo
Software Development/Quality Tools

19. Communication Dev
• Lead Dev Meeting
• AppDev Team Blog
• SharePoint
• Email Distribution Lists
• DOUG
QA
• QAD Meeting
• AppDev Team Blog
• QA Website
• QA SharePoint
• Email distribution lists
Communication Points

20. Learning Dev
•On-site Training
•Dev Pairing
•Pluralsight
•Website/Blogs
•Tech Conferences
•DOUG
•.NET/Other User Groups
QA
•On-site Training
•QAD/Dev Pairing
•Pluralsight
•Website/Blogs
•Tech Conferences
•Book Club
•DOUG
Learning Opportunities

21. Applied Testing (Testing Pyramid)
UI
Telerik
Integration
FitNesse
Postman
jMeter
Fortify on Demand (Security)
Unit
NUnit
Jasmine/Karma
FakeItEasy
NCrunch, Wallaby.js
Visibility via Jenkins

22. Jenkins
• Why Jenkins?
• Flexibility!
• Single Platform
• Highly configurable
• Automates the build and test (we use it for scheduling/running tests)
• Get source code from repository (VSTS)
• Dashboard - everyone can see what’s happening
• Generate report and notify stakeholders of build status
• Plug-in extensibility: Unit test coverage, last success and failure, trends

23. Jenkin’s Example Dashboard

24. QA Image
Target Install
• Telerik Runtime/ Chrome Plugin
• Postman(Node/Newman)
• jMeter
• NUnit
Common Software
• GIT
• Jenkins
• Beyond Compare
• Notepad++
Group Policies
• Auto-login with test account, Disable screen lock, UAC
• Browser settings, Screen resolution
Base Win 10 Image
• Includes (Java, Nuance PDF)

25. Summary
• The problem: No unified approach to quality
• Solution:
• Adopted shift-left approach to software development and quality
• Implemented the testing pyramid
• Testing architecture using Jenkins
• Increased collaboration and communication between QA and Devs
• Impact:
• Friction leads to lower adoption rates
• We increased test automation from 400 tests suites per day 2,300
• Standardized on a common set of tools
• Increased collaboration across teams
• Increased confidence in delivering software

26. What’s next ?
• Continue to focus on enterprise health & visibility
• Further grow our shift-left practices
• Automate our test data scenarios
• Grow the quality community
• Deliver great, high-quality software and experiences

27. Questions or
Thoughts?