Gov 2.0: Scaling, Automation, & Management in the Cloud

1. Scaling in the Cloud
Speaker:
Jesse Robbins CEO
‣ jesse@opscode.com
‣ @jesserobbins
‣ www.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 1

2. Opscode makes a new kind of
Infrastructure Automation,
offered as a hosted Service.
Copyright © 2010 Opscode, Inc. – Confidential – Do Not Redistribute 2

3. http://www.flickr.com/photos/
timyates/2854357446/sizes/l/

4. • Developers?
http://www.flickr.com/photos/
timyates/2854357446/sizes/l/

5. • Developers?
• Systems Administrators?
http://www.flickr.com/photos/
timyates/2854357446/sizes/l/

6. • Developers?
• Systems Administrators?
• Executives/Leaders?
http://www.flickr.com/photos/
timyates/2854357446/sizes/l/

7. For Developers...

8. For Developers...
• Do it yourself.

9. For Developers...
• Do it yourself.
• The infrastructure is the application
(and vice versa).

10. For Developers...
• Do it yourself.
• The infrastructure is the application
(and vice versa).
• You are not a Systems Administrator.

11. For Developers...
• Do it yourself.
• The infrastructure is the application
(and vice versa).
• You are not a Systems Administrator.
• You need tools.

12. Sysadmins..
http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft

13. Sysadmins..
• Say “Yes”.
http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft

14. Sysadmins..
• Say “Yes”.
• You never liked rack
and stack that much
anyway.
http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft

15. Sysadmins..
• Say “Yes”.
• You never liked rack
and stack that much
anyway.
• You have never
been more critical.
http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft

16. Sysadmins..
• Say “Yes”.
• You never liked rack
and stack that much
anyway.
• You have never
been more critical.
• Lean into it.
http://covers.oreilly.com/images/9780596007836/lrg.jpg
Lean into it appears courtesy of Cliff Moon, of Dynomite fame: http://twitter.com/moonpolysoft

17. Executives...

18. Executives...
• Not a magic unicorn

19. Executives...
• Not a magic unicorn
• Benefits come from efficiency, not raw Capex

20. Executives...
• Not a magic unicorn
• Benefits come from efficiency, not raw Capex
• Has real cultural implications at every level

21. Executives...
• Not a magic unicorn
• Benefits come from efficiency, not raw Capex
• Has real cultural implications at every level
• You are the biggest asset to success

22. “Traditional” Operations Operations - The “Secret Sauce”
50 50
40 40
# of Hours
30 30
20 20
Hardware
OS Install
10 10 Config
Upkeep
20 20
15 15
Servers
10 10 New
5 5 Existing
0 0
1 2 3 4 5 6 7 9 10 11 12 1 2 3 4 5 6 7 9 10 11 12
Week # Week #
(http://radar.oreilly.com/archives/2007/10/operations-advantage.html)
Copyright © 2010 Opscode, Inc - All Rights Reserved 7

23. “Traditional” Operations Operations - The “Secret Sauce”
50 50
This is the secret of
40 40
Cloud Computing.
Every other virtue stems from
# of Hours
30 30
here.
20 20
Hardware
OS Install
10 10 Config
Upkeep
20 20
15 15
Servers
10 10 New
5 5 Existing
0 0
1 2 3 4 5 6 7 9 10 11 12 1 2 3 4 5 6 7 9 10 11 12
Week # Week #
(http://radar.oreilly.com/archives/2007/10/operations-advantage.html)
Copyright © 2010 Opscode, Inc - All Rights Reserved 7

25. You are 10% Unique

26. You are 10% Unique
And itʼs probably
the things you did wrong

27. Infrastructure is Hard
Copyright © 2010 Opscode, Inc - All Rights Reserved 9

28. Infrastructure is Hard
1999
Inventory, packaged file transers and desktops
Copyright © 2010 Opscode, Inc - All Rights Reserved 9

29. Infrastructure is Hard
1999
Inventory, packaged file transers and desktops
2005
Unattended bare metal servers “very very” hard
7k Nodes took 5 days w/90 success
Copyright © 2010 Opscode, Inc - All Rights Reserved 9

30. Infrastructure is Hard
1999
Inventory, packaged file transers and desktops
2005
Unattended bare metal servers “very very” hard
7k Nodes took 5 days w/90 success
2007
Unattended bare metal in under 10 minutes
Fully configured in under 3 mins
Copyright © 2010 Opscode, Inc - All Rights Reserved 9

31. Infrastructure is Hard
1999
Inventory, packaged file transers and desktops
2005
Unattended bare metal servers “very very” hard
7k Nodes took 5 days w/90 success
2007
Unattended bare metal in under 10 minutes
Fully configured in under 3 mins
2008
Unattended server in 2 minutes
5000 servers in a week
Copyright © 2010 Opscode, Inc - All Rights Reserved 9

32. Infrastructure is Hard
1999
Inventory, packaged file transers and desktops
2005
Unattended bare metal servers “very very” hard
7k Nodes took 5 days w/90 success
2007
Unattended bare metal in under 10 minutes
Fully configured in under 3 mins
2008
Unattended server in 2 minutes
5000 servers in a week
2010
10k Nodes in under 5 minutes
Copyright © 2010 Opscode, Inc - All Rights Reserved 9

33. Infrastructure is changing
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

34. Infrastructure is changing
‣ Easier to get (good!)
...but harder to manage (bad!)
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

35. Infrastructure is changing
‣ Easier to get (good!)
...but harder to manage (bad!)
‣ Demand is dynamic
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

36. Infrastructure is changing
‣ Easier to get (good!)
...but harder to manage (bad!)
‣ Demand is dynamic
‣ Developers are crucial to Operations
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

37. Infrastructure is changing
‣ Easier to get (good!)
...but harder to manage (bad!)
‣ Demand is dynamic
‣ Developers are crucial to Operations
‣ Web / Cloud services are proliferating
...and Enterprise is following along.
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

38. Infrastructure is changing
‣ Easier to get (good!)
...but harder to manage (bad!)
‣ Demand is dynamic
‣ Developers are crucial to Operations
‣ Web / Cloud services are proliferating
...and Enterprise is following along.
‣ Manual configuration no longer a crutch
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

39. Infrastructure is changing
‣ Easier to get (good!)
...but harder to manage (bad!)
‣ Demand is dynamic
‣ Developers are crucial to Operations
‣ Web / Cloud services are proliferating
...and Enterprise is following along.
‣ Manual configuration no longer a crutch
‣ Few tools to solve a ubiquitous problem
Copyright © 2010 Opscode, Inc - All Rights Reserved 10

40. Managing Infrastructure Is Hard
Has Always Been
Proprietary Solutions
Previous Attempts Typically...
1980
• Solve very little of the problem...
1989 • Reach just a handful of large,
enterprise customers
1999 • Require custom implementations with
large professional services bills
• Deployed exclusively on-premise
2001
• Acquired by companies with large
consulting organizations (IBM, HP, CA)
Copyright © 2010 Opscode, Inc - All Rights Reserved

41. Google, Amazon, Microsoft
built their own tools
Copyright © 2010 Opscode, Inc - All Rights Reserved 12

42. but it’s “secret sauce”
Copyright © 2010 Opscode, Inc - All Rights Reserved 13

43. P
everyone else is here
... inexperienced & poorly
equipped for the world they
must now operate in.
Copyright © 2010 Opscode, Inc - All Rights Reserved 14

44. “Cloud”

45. Alistair’s mom’s definition
Cloud

46. Alistair’s mom’s definition
Cloud = Web

47. Alistair’s mom’s definition
Cloud = Web = Internet

48. Alistair’s mom’s definition
Cloud = Web = Internet = Useless

49. Slide courtesy Alistair Croll - alistair@rednod.com

50. Private Public
Slide courtesy Alistair Croll - alistair@rednod.com

51. Private Public
Managed
Virtualization
hosting
Slide courtesy Alistair Croll - alistair@rednod.com

52. Private Public
IaaS IaaS
Managed
Virtualization
hosting
Slide courtesy Alistair Croll - alistair@rednod.com

53. Private Public
PaaS PaaS
IaaS IaaS
Managed
Virtualization
hosting
Slide courtesy Alistair Croll - alistair@rednod.com

54. Private Public
SaaS
PaaS PaaS
IaaS IaaS
Managed
Virtualization
hosting
Slide courtesy Alistair Croll - alistair@rednod.com

55. Private Public
nt t o SaaS
a
w d s,
o u lo u
y c
If l k
PaaS
rs t.PaaS
t a ne f i
i ck o
IaaS p IaaS
Managed
Virtualization
hosting
Slide courtesy Alistair Croll - alistair@rednod.com

56. Infrastructure as a Service
(IaaS)
Amazon EC2, Rackspace Cloud, Terremark,
Gogrid, Joyent (and nearly every private
cloud built on Zenserver or VMWare.)
Slide courtesy Alistair Croll - alistair@rednod.com

57. Dedicated On-premise Virtual Third-party
hardware private clouds private clouds public clouds
Slide courtesy Alistair Croll - alistair@rednod.com

58. Slide courtesy Alistair Croll - alistair@rednod.com

59. Always on
premise
Private
Compliance-
enforced
Need to track and
audit
Legislative
Data near local
computation
Slide courtesy Alistair Croll - alistair@rednod.com

60. Always on Can be done
premise anywhere
Private
Compliance- Testing
enforced
Training
Need to track and
Prototyping
audit
Batch processing
Legislative
Seasonal load
Data near local
computation
Slide courtesy Alistair Croll - alistair@rednod.com

61. Always on Can be done Always in
premise anywhere cloud
Private
Partner access
Compliance- Testing
enforced Proximity to cloud
Training services (storage,
Need to track and
Prototyping CDN, etc.)
audit
Batch processing Massively grid/
Legislative
Seasonal load parallel (genomic,
Data near local modelling)
computation
Slide courtesy Alistair Croll - alistair@rednod.com

62. Always on Can be done Always in
premise anywhere cloud
Load/pricing engine
Private
Partner access
Compliance- Testing
enforced Proximity to cloud
Training services (storage,
Need to track and
Prototyping CDN, etc.)
audit
Batch processing Massively grid/
Legislative
Seasonal load parallel (genomic,
Data near local modelling)
computation
Slide courtesy Alistair Croll - alistair@rednod.com

63. Always on Can be done Always in
premise anywhere cloud
Load/pricing engine
Private
Partner access
Compliance- Testing
enforced Proximity to cloud
Training services (storage,
Policy engine
Need to track and
Prototyping CDN, etc.)
audit
Batch processing Massively grid/
Legislative
Seasonal load parallel (genomic,
Data near local modelling)
computation
Slide courtesy Alistair Croll - alistair@rednod.com

64. Virtual machine
(infrastructure cloud)
Always on Can be done Always in
premise anywhere cloud
Load/pricing engine
Private
Partner access
Compliance- Testing
enforced Proximity to cloud
Training services (storage,
Policy engine
Need to track and
Prototyping CDN, etc.)
audit
Batch processing Massively grid/
Legislative
Seasonal load parallel (genomic,
Data near local modelling)
computation
Slide courtesy Alistair Croll - alistair@rednod.com

65. Compute task
(service cloud)
Always on Can be done Always in
premise anywhere cloud
Load/pricing engine
Private
Partner access
Compliance- Testing
enforced Proximity to cloud
Training services (storage,
Policy engine
Need to track and
Prototyping CDN, etc.)
audit
Batch processing Massively grid/
Legislative
Seasonal load parallel (genomic,
Data near local modelling)
computation
Slide courtesy Alistair Croll - alistair@rednod.com

66. Automation

67. Bootstrapping

68. Bootstrapping Approaches
Good Bad Time
Known Costs, No High Waste (Hoarding)
Variation. Red Tape
Corp Approvals Anything you want, as long Expensive ($/Time) 6-8w
as IT pre-approved it. Long lead time
Lower Waste
Agile Corp Known Costs. Less Red Tape
Total Hardware Control. Still slow 2-4w
Approvals Trivial Approvals. Expensive ($/Time)
Shorter lead time
Variable Costs.
Highly Adaptable. Variable Costs.
Cloud Minimal lead time.
Trivial approvals.
No control over hardware.
Must re-train.
5-10m
No humans needed.

69. Configuration
curl -O http://brainspl.at/velocity.sh && sh velocity.sh

70. Configuration Approaches
Good Bad
Slow.
You can do anything.
Error Prone (Bus Error!)
Manual Results in an intimate knowledge
of the details.
Non-repeatable.
Difficult knowledge transfer.
Rarely idempotent.
More repeatable.
Hard to collaborate.
Ad-Hoc Knowledge is dispersed.
Built your way, with your model.
Brittle.
No API.
Repeatable.
Infrastructure Idempotent.
Agile.
Have to learn how to use it.
Hard things remain hard.
as Code Sharable.
Self documenting.
Not magic. (Yet!)

71. Command and
Control

72. Command and Control
Good Bad
Super flexible. Error Prone.
Can do almost anything. Slow.
Meatcloud* Always easy to find someone to
blame.
Expensive to Scale.
Not repeatable.
Free will. Free will.
One-off by neccessity.
More repeatable.
Tooling sprawl.
Ad-Hoc Easier to scale.
Less error prone (hopefully!)
Hard to share solutions.
Much higher learning curve.
One system to learn.
Scales well. Not everything maps cleanly.
Framework Paint by numbers.
Repeatable.
Trades depth of knowledge for
ease of use.
Two-Way.
*Meatcloud appears in this presentation courtesy of Andrew Shafer - http://is.gd/Ega

73. Lightning Strikes!
DOOM
Webservers
Database Servers
Webservers

74. Lightning Strikes!
DOOM
X
Webservers
XX
Database Servers
Webservers

75. Lightning Strikes!
DOOM
X 1
1 2
1
Signals Moar!
XX
Monitoring Command & Bootstrapping
System Webservers
Updates Control
Provisions
2
1
3
1
5
1 3
1
Database Servers
4
1 4
1
Configuration
Webservers

76. Lightning Strikes!
Monitoring
Signals Nanite
/node/down
Service
DOOM
X 1
1 2
1
Signals Moar!
XX
Monitoring Command & Bootstrapping
System Webservers
Updates Control
Provisions
2
1
3
1
5
1 3
1
Database Servers
4
1 4
1
Configuration
Webservers

77. Lightning Strikes! Nanite
boots new EC2
Nanite removes DOOM
Instances, with
Chef Role +
nodes in Chef Attribute
X 1
1 2
1
Signals Moar!
XX
Monitoring Command & Bootstrapping
System Webservers
Updates Control
Provisions
2
1
3
1
5
1 3
1
Database Servers
4
1 4
1
Configuration
Webservers

78. Lightning Strikes!
DOOM
X 1
1 2
1
Provisions
Instances, EBS,
Signals Moar! Elastic IPs
XX
Monitoring Command & Bootstrapping
System Webservers
Updates Control
Provisions
2
1
3
1
5
1 3
1
Database Servers
4
1 4
1
Configuration
Webservers

79. Lightning Strikes!
DOOM
X 1
1 2
1
Signals Moar!
XX
Monitoring Command & Bootstrapping
System Webservers
Updates Control
Provisions
2
1
3
1
5
1 3
1
Chef Database Servers
configures nodes 4
1 4
1
according to Configuration
assigned
Webservers

80. Lightning Strikes!
DOOM
Chef
X 1
1 2
1
Signals Moar!
updates the
XX
Monitoring Command & Bootstrapping
monitoring System Webservers
Updates Control
Provisions
system 5
1
2
1
3
1
3
1
Database Servers
4
1 4
1
Configuration
Webservers

81. A word about Scaling...

82. Typical Peak Load
1.Bring on capacity as traffic ramps up
2.Take down capacity as it ramps down
3.10-15 Minutes on either side, fully
unattended
Graphs in this portion of the presentation taken from Theo Schlossnagle
http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes

83. Atypical Load
No way However,
around you are
Capacity still better
Planning off!
1.Hope you know it is coming.
2.Increase capacity in advance.
3.Take down capacity as it ramps down.
Graphs in this portion of the presentation taken from Theo Schlossnagle
http://omniti.com/seeds/dissecting-todays-internet-traffic-spikes

84. Capacity Planning is
king.
http://www.flickr.com/photos/allspaw/2095439645/sizes/l/

86. Have a queue?

87. Have a queue?
Does it scale linearly with more resources?

88. Have a queue?
Does it scale linearly with more resources?
Congratulations - you can auto-scale!

89. NoSQL
http://www.flickr.com/photos/wingler/3429634150/sizes/l/

90. CAP Theorem
• Consistency Pick
Two
• Availability
• Partition Tolerance

91. Most SQL Databases
• Choose Consistency over all
• Availability comes distant second

92. Web Applications need...
• Availability
• Partition Tolerance

93. “Global temporal
consistency is a fiction”
Christopher Brown

94. Choosing Consistency
for your Web App...
Means failure is global

95. When you choose
Partition Tolerance and
Availability...
You fail or succeed for a subset of
users

96. Apologies
• Apologize after the fact for failures
• Better than nothing at all

97. NoSQL
• Many different tools
• They tweak CAP differently
• CouchDB
• Cassandra
• Redis
• MongoDB

98. Scaling in the Cloud
Speaker:
Jesse Robbins CEO
‣ jesse@opscode.com
‣ @jesserobbins
‣ www.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 43