Kevin Smith is the Director of Server Engineering at Opscode and has been developing software for 17 years including 7 years with Erlang. He discusses infrastructure as code, configuration management with Chef, and how Chef can be used in large environments. Specifically, he covers how Chef uses recipes, roles, attributes and resources to declaratively configure nodes. He also discusses how the Chef server and clients interact and how search is used. Finally, he notes how Chef is open source and has a large community contributing cookbooks and tools to support deployments of all sizes.

1. An Extemporaneous Introduction
To Chef
Kevin A. Smith
Director of Server Engineering

2. Who am I?
• Director of Server Engineering @ Opscode
• Software developer for 17 years
• 7 years with Erlang
• Alumni of DCRI, SAS, Red Hat, Basho
• Erlang In Practice @ PragProg

3. Agenda
• Infrastructure as Code
• Configuration Management
• Chef 101
• Chef in Large Environments
http://www.flickr.com/photos/koalazymonkey/3590953001/

4. Infrastructure as Code

5. Building and
managing
infrastructure
programmatically
Infrastructure as Code

6. Enable the reconstruction
of the business from
nothing but a source code
repository, an application
data backup, and bare
metal resources.
Infrastructure as Code

7. Configuration
Management

8. The Old Way

9. Manual Configuration
• Labor intensive
• Error prone
• Hard to reproduce

10. Scripting
• Very brittle
• Throw away, one off scripts
• grep sed awk perl
• curl | bash

11. File Distribution
• NFS mounts
• rdist
• scp-on-a-for-loop
• rsync on cron

12. This does not scale!
for i in `cat servers.txt` ; do scp ntp.conf root@$i:/etc/
ntpd.conf ; done
for i in `cat servers.txt` ; do ssh root@$i /etc/init.d/ntpd
restart ; done
for i in `cat servers.txt` ; do ssh root@$i chkconfig ntpd
on ; done

13. See nodes grow.
Load
Balancer
Application
Server
Database
Application
Server

14. Load
Balancer
Application
Server
Database
Application
Server
Load
Balancer
Database
Grow, nodes. Grow!

15. Datacenter #1
Load
Balancer
App
Server
Database
App
Server
Load
Balancer
Database
Datacenter #2
Load
Balancer
App
Server
Database
App
Server
Load
Balancer
Database
Internet
There are a lot of nodes!

16. A New Way

17. Declarative Configuration
• Define policy
• Say what, not how
• Abstract interface to resources

18. Idempotence
• Property of a
declarative interface
• f(x) = x
• Eliminates brittleness
• Safe to run over and
over
package "ntp" do
action :install
end
template "/etc/ntp.conf" do
source "ntp.conf.erb"
owner "root"
group "root"
mode 0644
notifies :restart, "service[ntpd]"
end
service "ntpd" do
action [:enable,:start]
end

19. Convergence
• Running an agent “converges”
a system onto desired state
• Fights entropy and
unauthorized changes
• Update function inputs to deal
with changing requirements
$ echo “boom” > /etc/ntp.conf
$ chef-client
$ grep server /etc/ntp.conf | head -n 1
us.pool.ntp.org
$ ps -e | grep ntp
1799 ? 00:00:00 ntpd
$ /etc/init.d/ntpd stop
$ chef-client
ps -e | grep ntp
1822 ? 00:00:00 ntpd

20. Chef 101
http://www.flickr.com/photos/lapstrake/2711240606/in/photostream/

21. The chef-client runs on your
systems.

22. Clients talk to a Chef server.

23. Client server conversations
are protected with SSL and
RSA signatures.

24. Each system running Chef is
called a Managed Node.

25. Chef API
Server
RDBMS
Search
Engine
Asset
Store
Managed Node
Chef
Client
System Architecture

26. Nodes have attributes
{
"kernel": {
"machine": "x86_64",
"name": "Darwin",
"os": "Darwin",
"version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT
2010; root:xnu-1504.7.4~1/RELEASE_I386",
"release": "10.4.0"
},
"platform_version": "10.6.4",
"platform": "mac_os_x",
"platform_build": "10F569",
"domain": "local",
"os": "darwin",
"current_user": "mray",
"ohai_time": 1278602661.60043,
"os_version": "10.4.0",
"uptime": "18 days 17 hours 49 minutes 18 seconds",
"ipaddress": "10.13.37.116",
"hostname": "morbo",
"fqdn": "morbomorbo.local",
"uptime_seconds": 1619358
}
Platform
Kernel
Hostname, etc.

27. Node attributes are
searchable.
$ knife search node ‘platform:mac_os_x’
search(:node, ‘platform:mac_os_x’)

28. Nodes have “to do” lists.

29. Nodes have a Run List
% knife node show hadoop-prod.example.com -r
{
"run_list": [
"role[base]",
"role[hadoop-worker]"
]
}

30. Nodes can have Roles.

31. Aspirational Roles
• webserver
• database_master
• monitoring
• hadoop-worker

32. Roles have Attributes
and a run list.

33. Roles
name "hadoop-worker"
description "Hadoop cluster member”
run_list(
"role[base]",
"recipe[java]",
"recipe[hadoop]",
“recipe[hadoop-config]"
)
default_attributes(
"hadoop-config" => {
"config_path" => “/etc/hadoop”
}
)

34. chef-client configures
resources on managed nodes.
cookbook_file
template
service
package
deploy
git
http_request
link
ruby_block
log
bash
execute
remote_file
user

35. Chef Resources
• Have a type.
• Have a name.
• Have parameters.
• Take action to put the resource
in the declared state.
• Can send notifications to other
resources.
package "apache2" do
action :install
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
notifies :restart, "service[apache2]"
end
service "apache2" do
supports :restart => true
action [:enable, :start]
end

36. package “hadoop”
{yum install hadoop
apt-get install hadoop
pacman sync hadoop
pkg_add -r hadoop
Chef Providers

37. Recipes are collections
of resources.

38. Chef Recipes
• Resources are evaluated in the
order they appear.
package "haproxy" do
action :install
end
template "/etc/haproxy/haproxy.cfg" do
source "haproxy.cfg.erb"
owner "root"
group "root"
mode 0644
notifies :restart, "service[haproxy]"
end
service "haproxy" do
supports :restart => true
action [:enable, :start]
end

39. Chef Recipes
• Recipes can include other
recipes.
• Included recipes are also
evaluated in order.
include_recipe "apache2"
include_recipe "apache2::mod_rewrite"
include_recipe "apache2::mod_deflate"
include_recipe "apache2::mod_headers"
include_recipe "apache2::mod_php5"

40. Chef Recipes
• Extend recipes with
Ruby.
%w{ php5 php5-dev php5-cgi }.each do |pkg|
package pkg do
action :install
end
end

41. • Dynamic configuration
through search.
pool_members = search("node", "role:app_server")
template "/etc/haproxy/haproxy.cfg" do
source "haproxy.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members
notifies :restart, "service[haproxy]"
end
Chef Recipes

42. Cookbooks are packages for
recipes and related files.

43. Cookbook Metadata
maintainer "Opscode, Inc."
maintainer_email "cookbooks@opscode.com"
license "Apache 2.0"
description "Installs/Configures tomcat"
long_description IO.read(File.join(File.dirname(__FILE__),
'README.md'))
version "0.10.3"
%w{ java jpackage }.each do |cb|
depends cb
end
%w{ debian ubuntu centos redhat fedora }.each do |os|
supports os
end
recipe "tomcat::default", "Installs and configures Tomcat"

44. Cookbooks are Source Code
% git log
commit d640a8c6b370134d7043991894107d806595cc35
Author: jtimberman <joshua@opscode.com>
Import nagios version 1.0.0
commit c40c818498710e78cf73c7f71e722e971fa574e7
Author: jtimberman <joshua@opscode.com>
installation and usage instruction docs
commit 99d0efb024314de17888f6b359c14414fda7bb91
Author: jtimberman <joshua@opscode.com>
Import haproxy version 1.0.1
commit c89d0975ad3f4b152426df219fee0bfb8eafb7e4
Author: jtimberman <joshua@opscode.com>
add mediawiki cookbook
commit 89c0545cc03b9be26f1db246c9ba4ce9d58a6700
Author: jtimberman <joshua@opscode.com>
multiple environments in data bag for mediawiki

45. OSS & Community Oriented
• Apache 2.0 License
• Wiki, mailing lists, shared cookbook repos
• http://community.opscode.com
• Healthy ecosystem
• 20k+ users
• Hundreds of contributors
• Community tooling: Food Critic,Test Kitchen, Berkshelf

46. Chef In
“Large” Environments

47. New Server
• Ground up rewrite Ruby/C Erlang
• Order of magnitude more scalable
• 2k nodes 20k+ nodes per server*
*Depending on specific work load

48. High Scalability Users
• Facebook
• Cycle Computing
• edmunds.com

49. Push Execution
• Converge infrastructure on demand
• Real-timey view of managed infrastructure
• Reduces change latency
• 4k nodes now, 10k soon

50. Network Automation
• Network provisioning and configuration
• VLANs, QoS, etc.
• Partnered w/Arista on PoC (Fall 2012)
• More coming soon!

51. ThankYou