Buildout talk
- 2. Formspring helps people find out more about each
other by sharing interesting & personal responses
Monday, September 19, 2011
- 7. Memcache
Application
Monday, September 19, 2011
- 8. Memcache
Redis
Application
Monday, September 19, 2011
- 9. Memcache
Redis
Application
MySQL
Monday, September 19, 2011
- 10. Memcache
Redis
Application
MySQL
Cassandra
Monday, September 19, 2011
- 11. Memcache
Redis SQS
Application
MySQL
Cassandra
Monday, September 19, 2011
- 12. Memcache
Redis SQS
Application
MySQL Kestrel
Cassandra
Monday, September 19, 2011
- 13. PHP
PHP
PHP
PHP
QWPHP
QW
QW
QW
QW
Memcache
Redis SQS
Application
MySQL Kestrel
Cassandra
Monday, September 19, 2011
- 14. PHP
PHP
PHP
PHP
QWPHP
QW
QW
QW
QW
Memcache
Redis SQS
Application
MySQL Kestrel
Cassandra
Python
Python
Python
Python
Python
QW
QW
QW
QW
QW
Monday, September 19, 2011
- 15. Memcache
Redis
Python
Python
Python
Python
Python
QW
QW
QW
QW Kestrel
MySQL QW
Cassandra
Monday, September 19, 2011
- 16. Python
Memcache QW
Redis
Kestrel
MySQL
Python
Cassandra QW
Monday, September 19, 2011
- 17. Python
Memcache QW
Redis
Kestrel
MySQL
Python
Cassandra QW
Monday, September 19, 2011
- 18. Python
Memcache QW
Redis
Kestrel
MySQL
Python
Cassandra QW
Monday, September 19, 2011
- 25. Workers.py
class StreamPush:
...
class SmileOnAnswers:
...
class SmileConnection:
...
class Inbox:
...
Monday, September 19, 2011
- 26. Workers.py
• ./python2.6 kestrel_worker :
• Thread Count
• Queue Name
• Class Name
Monday, September 19, 2011
- 27. Deploy
Queue-Stream
code.tar.gz Queue-Smiles
Queue-Inbox
Monday, September 19, 2011
- 28. Problems
• Big archive
Monday, September 19, 2011
- 29. Problems
• Big archive
• Dependencies
Monday, September 19, 2011
- 30. Problems
• Big archive
• Dependencies
• Package AMI
Monday, September 19, 2011
- 31. Problems
• Big archive
• Dependencies
• Package AMI
• Difficult to maintain / collaborate
Monday, September 19, 2011
- 32. What needs to be done
• Organize Codebase
Monday, September 19, 2011
- 33. What needs to be done
• Organize Codebase
• Split in folders / packages
Monday, September 19, 2011
- 34. What needs to be done
• Organize Codebase
• Split in folders / packages
• Manage dependencies automatically
Monday, September 19, 2011
- 36. Buildout
• No installation needed
Monday, September 19, 2011
- 37. Buildout
• No installation needed
• 3 easy steps
Monday, September 19, 2011
- 38. Buildout
• Download bootstrap.py
Monday, September 19, 2011
- 39. Buildout
• Download bootstrap.py
• Run boostrap.py
Monday, September 19, 2011
- 40. Buildout
• Download bootstrap.py
• Run boostrap.py
• Run bin/buildout
Monday, September 19, 2011
- 42. Buildout Recipe
• Plugin mechanism
• Main Recipes
Monday, September 19, 2011
- 43. Buildout Recipe
• Plugin mechanism
• Main Recipes
• zc.recipe.egg
Monday, September 19, 2011
- 44. Buildout Recipe
• Plugin mechanism
• Main Recipes
• zc.recipe.egg
• zc.recipe.cmmi
Monday, September 19, 2011
- 45. Buildout Recipe
• Plugin mechanism
• Main Recipes
• zc.recipe.egg
• zc.recipe.cmmi
• djangorecipe
Monday, September 19, 2011
- 46. Organize Codebase
|-- fisher
| |-- bootstrap.py
| |-- buildout.cfg
| |-- setup.py
| `-- src
| `-- fisher
| `-- __init__.py
|-- flake
| |-- bootstrap.py
| |-- buildout.cfg
| |-- setup.py
| `-- src
| `-- flake
| |-- README
| |-- __init__.py
| |-- flake.py
| `-- static
| |-- favicon.ico
| `-- robots.txt
|-- kestrel
| |-- bootstrap.py
| |-- buildout.cfg
| |-- setup.py
| `-- src
| `-- kestrel
Monday, September 19, 2011
- 67. Advantages
• Flexibility with dependencies.
Monday, September 19, 2011
- 68. Advantages
• Flexibility with dependencies.
• Build your own eggs.
Monday, September 19, 2011
- 69. Advantages
• Flexibility with dependencies.
• Build your own eggs.
• Test things !
Monday, September 19, 2011
- 71. Deploy
Redis Queue-Stream
Kestrel
stream.tar.gz Queue-Smiles
inbox.tar.gz
smiles.tar.gz
Queue-Inbox
Monday, September 19, 2011
- 72. Deploy
Queue-Stream
Kestrel
Memcache
PyYAML Queue-Smiles
stream.tar.gz
inbox.tar.gz
smiles.tar.gz
Queue-Inbox
Monday, September 19, 2011
- 73. Deploy
Queue-Stream
stream.tar.gz Fisher Queue-Smiles
inbox.tar.gz
smiles.tar.gz Kestrel
Queue-Inbox
Monday, September 19, 2011
- 74. Advantages
• Targeted deploys.
Monday, September 19, 2011
- 75. Advantages
• Targeted deploys.
• Less failures.
Monday, September 19, 2011
- 76. Advantages
• Targeted deploys.
• Less failures.
• Less work for Ops.
Monday, September 19, 2011
- 77. Advantages
• Targeted deploys.
• Less failures.
• Less work for Ops.
• Manage your versions.
Monday, September 19, 2011
- 79. Downtime ?
• Recipe to proxy PyPi.
Monday, September 19, 2011
- 80. Downtime ?
• Recipe to proxy PyPi.
• Cronjob.
Monday, September 19, 2011
- 81. Downtime ?
• Recipe to proxy PyPi.
• Cronjob.
• Local PyPi.
Monday, September 19, 2011
- 83. So .. Buildout
• Manage dependencies
• Test libraries on the fly
Monday, September 19, 2011
- 84. So .. Buildout
• Manage dependencies
• Test libraries on the fly
• Easy to scale / deploy
Monday, September 19, 2011
- 85. So .. Buildout
• Manage dependencies
• Test libraries on the fly
• Easy to scale / deploy
• Highly Recommended =)
Monday, September 19, 2011
- 86. Questions ?
• Ask me on Formspring
• formspring.me/martincozzi
• formspring.me/formspringEng
Monday, September 19, 2011
- 87. We are hiring
• Ops
• Front-end
• Back-end
martin@formspring.me
rob@formspring.me
Monday, September 19, 2011
