More Related Content Similar to Buildout: How to maintain big app stacks without losing your mind (20) Buildout: How to maintain big app stacks without losing your mind1. Buildout How to maintain big app stacks without losing your mind Dylan Jay Sypy.org 3. “It should be possible to check-in a buildout specification and reproduce the same software” 12. So what is buildout? Gaffer tape is like the force. It has a light side, a dark side, and it holds the universe together. 18. Anatomy of a Buildout buildout.cfg: [buildout] parts = MyPart [MyPart] recipe = recipepackage arg1 = value1 19. Initialising $ easy_install zc.buildout … Finished processing dependencies for zc.buildout $ buildout init Creating directory '/Users/dylanjay/Projects/sandpit/buildout/bin'. Creating directory '/Users/dylanjay/Projects/sandpit/buildout/parts'. Creating directory '/Users/dylanjay/Projects/sandpit/buildout/develop-eggs'. Generated script '/Users/dylanjay/Projects/sandpit/buildout/bin/buildout'. $ bin/buildout 21. Hello World [buildout] parts = helloworld chmod [helloworld] recipe = collective.recipe.template output = ${buildout:bin-directory}/hello input = inline: echo 'hello world' [chmod] recipe = plone.recipe.command command = chmod +x ${helloworld:output} 22. Dependencies and Substitutions [ buildout] parts = chmod [ helloworld] recipe = collective.recipe.template output = ${buildout:bin-directory}/hello input = inline: echo 'hello world' [ chmod] recipe = plone.recipe.command command = chmod +x ${helloworld:output} 23. Installing Packages (easy_install) [buildout] parts = helloworld [helloworld] recipe = zc.recipe.egg eggs = Fabulous Pillow Interpreter = python $ bin/python -m fabulous.text hello world 24. Installing Scripts [ buildout] parts = helloworld [ helloworld] recipe = zc.recipe.egg eggs = fabulous Pillow # if package has its own entry-point, you don't need the stuff below initialization = from fabulous import text; print text.Text("Hello World", shadow=True) go = lambda: True scripts = hello entry-points = hello=__main__:go 25. Inside bin/script (not rocket science) #!/Users/dylanjay/Projects/virtual/buildout/bin/python import sys sys.path[0:0] = [ '/../download-cache/eggs/elementtree-1.2.7_20070827_preview-py2.4.egg', '/../download-cache/eggs/archetypes.kss-1.4.3-py2.4.egg', … ] import plone.recipe.zope2instance.ctl if __name__ == '__main__': plone.recipe.zope2instance.ctl.main( ["-C", '/Users/dylanjay/Projects/gcio/parts/instance/etc/zope.conf'] + sys.argv[1:]) 26. Inside Recipes class HelloRecipe(object): def __init__(self, buildout, name, options): self.input = options.get('input','world') self.output = options['output'] = 'hello %s' % self.input self.location = options['location'] = self.buildout['buildout']['bin-directory']+'hello' def install(self): # Return files that were created by the recipe. The buildout # will remove all returned files upon reinstall. f = open(self.location).write(self.output) ; f.close() return [self.location] def update(self): pass 27. mr.developer [buildout] parts = helloworld extensions = mr.developer auto-checkout= helloworld [sources] helloworld = git git://github.com/topher200/genetic-hello-world-python.git [ helloworld] recipe=zc.recipe.egg eggs = genetic-hello-world-python # WARNING: this example doesn't work 28. zc.recipe.cmmi [buildout] parts = instance varnish [instance] recipe = plone.recipe.zope2instance eggs = Plone HelloWorldPlone Http-address = 127.0.0.1:8080 [varnish-build] recipe = zc.recipe.cmmi url = http://downloads.sourceforge.net/varnish/varnish-2.1.3.tar.gz [varnish] recipe = plone.recipe.varnish daemon = ${varnish-build:location}/sbin/varnishd bind = 127.0.0.1:8080 backends = ${instance:http-address} 29. Versions [ buildout] parts = instance varnish [ instance] recipe = plone.recipe.zope2instance eggs = Plone >= 4.1 30. Versions Pinning [ buildout] parts = instance [ instance] recipe = plone.recipe.zope2instance eggs = Plone [ versions] Plone= 4.1 31. Versions KGS [ buildout] extends = http://dist.plone.org/release/4.1/versions.cfg parts = instance varnish [ instance] recipe = plone.recipe.zope2instance eggs = Plone 32. Version Conflicts Installing. Getting section zeoserver. Initializing part zeoserver. Error: There is a version conflict. We already have: zope.component 3.8.0 but five.localsitemanager 1.1 requires 'zope.component<3.6dev'. Conflicts are caused by buildout having no way to know depenency specs until its too late. 33. Version Conflict Prevention $ bin/buildout -N (prevent auto upgrading packages) $ bin/buidout -v (discover where conflict arose) Or Pinning versions 34. dumppickedversions [buildout] extensions = buildout.dumppickedversions $ bin/buildout Getting distribution for 'buildout.dumppickedversions'. ... *************** PICKED VERSIONS **************** [versions] myegg = 1.1 setuptools = 2.1 zc.buildout = 1.5.3 zc.recipe.egg = 1.5.2 <BLANKLINE> *************** /PICKED VERSIONS *************** 35. Saving versions [ buildout] extensions = buildout.dumppickedversions dump-picked-versions-file = picked.cfg 36. Macros [ buildout] parts = instance varnish [ instance] recipe = plone.recipe.zope2instance eggs = Plone HelloWorldPlone http-address = 127.0.0.1:8080 [instance2] <=instance1 http-address = 127.0.0.1:8081 [ instance2] <= instance1 http-address = 127.0.0.1:8081 [ varnish-build] recipe = zc.recipe.cmmi url = http://downloads.sourceforge.net/varnish/varnish-2.1.3.tar.gz [ varnish] recipe = plone.recipe.varnish daemon = ${varnish-build:location}/sbin/varnishd bind = 127.0.0.1:80 backends = ${ instance:http-address} ${ instance1:http-address} ${ instance2:http-address} 37. mr.scripty [ ports_base] Instance1 = 80 Instance2 = 81 Instance3 = 83 [ ports] recipe=mr.scripty OFFSET = 1000 init= ... for key,value in self.buildout['ports_base'].items(): ... self.options[key] = str(int(value)+int(self.OFFSET)) 38. mr.scripty [ buildout] parts = instance varnish [ instance] recipe = plone.recipe.zope2instance eggs = Plone HelloWorldPlone http-address = ${ports:instance1} [ instance2] <= instance1 http-address = ${ports:instance2} [ instance2] <= instance1 http-address = ${ports:instance3} 40. Annotation mode $ bin/buildout annotate Annotated sections ================== [bfg] dependent-scripts= true /Users/dylanjay/Projects/sandpit/mobme/buildout.cfg eggs= repoze.bfg mobme /Users/dylanjay/Projects/sandpit/mobme/buildout.cfg index= http://dist.repoze.org/bfg/current/simple /Users/dylanjay/Projects/sandpit/mobme/buildout.cfg recipe= zc.recipe.egg /Users/dylanjay/Projects/sandpit/mobme/buildout.cfg [buildout] accept-buildout-test-releases= false DEFAULT_VALUE