The document describes a case study on using Chef to build a continuous delivery pipeline. It outlines setting up a minimum viable pipeline with development, verification, code review, integration testing, and promotion stages. Jenkins and Chef tools like Test Kitchen, Foodcritic, Berkshelf were used to automate building and testing code changes. The resulting pipeline provided smaller deployments, test-driven development, and greater confidence through automated verification and testing at each stage before code is promoted.

1. So Continuous.
Much Delivery.
Very Chef.
Wow.
A Case Study on using Chef to start building a
Continuous Delivery Pipeline

2. About Me
•
George Miranda
•
Sr Consultant at Chef Software, Inc.
•
Unix guy (15+ years)

6. Minimum Viable Pipeline

7. What we know
•
Step 1: Develop a new change
•
Step 2: ???
•
Step 3: Production!
•
MOAR FASTERZ

8. Case Study: Requirements
•
Must utilize existing tools within the company
•
Git for SCM
•
Jenkins approved for use
•
Working in a static VM environment
•
Just migrated to single cookbook repos
•
Starting with infrastructure cookbooks
•
Want a manual go-to-production button (ugh!)

9. Case Study: Code Review Model
•
Git PR model: branch from master for any
new feature
•
4-person team, only 3 active at any time
•
Code review done manually and informally
•
Simple communication/reqs (makes it easy!)

10. Figuring out new workflow
•
How are developers expected to work locally?
•
When do they push to remote? How do we
verify their work?
•
Code Review criteria: what does it mean to be
ready to merge?
•
How do we go from merged code to artifact?
•
How do we get that artifact all the way to
Production?

11. Local Development Work
•
New branch for every feature
•
Create a failing test
•
Write a resource to pass the test
•
Local commits
•
Test-Kitchen + guard
•
Once local tests passed, push to remote

12. Push to remote
•
Open a Pull Request (new branch to master)
•
Triggers a build via Jenkins GHPRB plugin

13. Push to remote
The Verify Build Job
•
Verify syntax (knife cookbook check)
•
Foodcritic Rules
•
Test-Kitchen w/ BATS busser

14. BATS: Simple Unit Tests
@test "My directory is created" {!
test -d /foo/bar!
}!
!
@test "A basharific test" {!
if [ foo != bar ]; then!
skip "foo isn't bar"!
fi!
!
run foo!
[ "$status" -eq 0 ]!
}!
!
•
•
https://github.com/sstephenson/bats
Super low learning curve (but also very limited)

15. Push to remote
•
If failed, notify
•
Another commit to the same branch
triggers another Verify Build Job
•
Super easy to track, comment, and approve
•
If passed, let’s go to Human Code Review

16. Human Code Review Rules
•
Only one change per one cookbook at one
time
•
Must have test for feature that changed
•
One for one: resource unit tests
•
Consider the smoke test

17. Unit Test vs Smoke Test
•
Unit tests: small, fast, check one single
concern
•
•
Smoke tests: test multiple things in the course
of one concern
•
•
In this context: checking Chef resources
In this context: check the intent of a recipe
Note: that was testing for this use case

18. When are we ready to merge?
•
Only 3 active team members at any given
time
•
•
•
Submitter cannot approve
Merge approval requires 2 approvals
Code review can happen at any time, but
only merge when you’re ready to fix it.

19. Merged code to artifact
•
Freeze your cookbooks!
•
Semantic versioning: Major.Minor.Patch
•
•
•
You own Major.Minor
The Pipeline owns .Patch
No one gets to knife upload
No one.!
Ever.!
•
"git merge" is the new "knife upload"

21. The Integration Job
•
Bumps Cookbook version
•
Re-commits to master
•
Upload frozen cookbook (via berks)
•
Pin that new cookbook to the Integration
environment
•
Converge all nodes that use that cookbook

22. The Integration Job
•
First sign that things may be broken
•
These nodes also run smoke tests
•
serverspec, minitest, etc

23. The Integration Job
•
We survived! Trigger the next job(s)
•
The Jenkins Build Pipelines Plugin allows
upstream/downstream definitions to string
together jobs
•
From here out, it’s all the same Promote Job*
•
After the Integration job, we just run X number
of Promote Jobs
* (mostly)

27. Promote Jobs
•
Pin cookbook to new Chef Environment
•
Converge all nodes using this cookbook
•
Run Tests

28. Pin the cookbook to Env
#!/opt/chef/embedded/bin/ruby
!
require 'chef/environment'
require 'chef'
Chef::Config.from_file("/var/lib/jenkins/tools/knife.rb")
!
def pin_env(env, cookbook_versions)
to = Chef::Environment.load(env)
cookbook_versions.each do |cb, version|
puts "Pinning #{cb} #{version} in #{env}"
to.cookbook_versions[cb] = version
end
to.save
end
!
cookbook_data = Array.new
!
if File.exists?(File.expand_path(File.join(ENV['WORKSPACE'], 'metadata.rb')))
metadata_file = File.expand_path(File.join(ENV['WORKSPACE'], 'metadata.rb'))
File.read(metadata_file).each_line do |line|
if line =~ /^names+["'](w+)["'].*$/
cookbook_data << $1
end
if line =~ /^versions+["'](d+.d+.d+)["'].*$/
cookbook_data << "= #{$1}"
end
end
end
!
cookbook_versions = Hash[*cookbook_data]
!
pin_env(ARGV[0], cookbook_versions)

29. Pin the cookbook to Env
$ berks apply <environment>

30. Converge Nodes
$ knife ssh "recipes:mycookbook AND
chef_environment:promote-environment”
'sudo chef-client'!
… OR …
Pushy!

31. Run Tests
•
Most testing frameworks have a Report
Handler to automatically run tests
•
chef-serverspec-handler
•
minitest-handler
•
Deploy to your nodes by adding
‘chef_handler’ to their run_list
•
Many community cookbooks are already
packaged with tests

32. Run Tests
•
In this particular use case:
•
Build job: BATS (unit tests)
•
Integration & Promote jobs: serverspec
(smoke tests)
•
UAT: also ran Cucumber tests (acceptance)

34. Promoting to more environments
•
Can string together N number of promotions
•
UAT
•
Production A
•
Production B
•
etc

37. Push to Production
•
In production monitoring is the test
•
Could not queue up changes reliably anyway
•
There is no spoon

38. Results
•
Small incremental deployments led to greater
confidence
•
TDD was pushed to the forefront of priorities
•
Commitment from Dev group to write
application deployment cookbooks
•
But the biggest lesson learned…

39. Let’s Go Devop with a CD tool
•
Continuous Delivery is a practice, not a tool
•
Small incremental changes in code
•
Small incremental changes in workflow
•
Small incremental changes in tooling
•
You will constantly improve your code, your
workflow, your tools, your team, and your
skills.

40. RECAP

41. What We Wanted
•
Step 1: Develop a new change
•
Step 2: ???
•
Step 3: Production!
•
MOAR FASTERZ

42. Wait… what was Step 2?
•
(Pre-req) Test Driven Development
•
2A. Establish development workflow before submitting changes *
•
2B. Auto verification of submission before humans look at it
•
2C. Humans Apply Code Review Criteria *
•
2D. Don’t merge unless you mean it *
•
2E. Merge kicks off an Integration Job
•
2F. Followed by a series of Promotion Jobs
•
2G. There is no spoon *

43. What We Got
•
•
Step 1: Develop a new change
Step 2:
(Pre-req) Test Driven Development
2A. Establish development workflow before submitting changes *
2B. Auto verification of submission before humans look at it
!
2C. Humans Apply Code Review Criteria *
2D. Don’t merge unless you mean it *
2E. Merge kicks off an Integration Job
!
2F. Followed by a series of Promotion Jobs
2G. There is no spoon *
•
Step 3: Production!
•
Step 4: Level Up. This is great!
•
Step 5: MOAR THINGS! Wait. This is hard!
•
Go to Step 1

44. Key Chef Ecosystem Tools
•
Test Kitchen — http://kitchen.ci/
•
Guard Plugin for Test Kitchen —
https://github.com/test-kitchen/guard-kitchen
•
Foodcritic — http://acrmp.github.io/foodcritic/
•
Berkshelf — http://berkshelf.com/

45. Helpful Jenkins Plugins
•
git
•
github
•
build-pipeline-plugin
•
ghprb
•
warnings
•
mailer

46. I want to hear from you!
!
@gmiranda23
gmiranda@getchef.com