How Rackspace is leveraging VCL with Varnishtest

1. LEVERAGING THE POWER OF VCL
AND VARNISHTEST
David Porter
Tech Lead / Software Developer – Global eCommerce
Varnish Summit Boston – March 2017

2. PURPOSE OF THIS TALK
To inspire VCL users to embrace software development practices for all to benefit.

3. THE PLAN• Where we were
• Where we are

4. WHERE WE WERE
F5s, NetSec, Tickets, Trouble

5. And it was not good.
5
• F5 Local Traffic Manager (LTM)
• Used as a load balancer, ACL, and a bit more…
• It's software (TCL) but no development practices.
‣ All handled by NetSec, tickets only, super slow
‣ Prone to typos, no linting
‣ No version control, full copy/paste, rapidly growing files
‣ No testing or measure to prevent regression
‣ Not much of a review process (code pasted in ticket)
‣ Too much duplication between environments, nothing shared
In the beginning was the F5…

6. A lot more.
6
• Broaden the team and contributor base
‣ Insight into logic
‣ Insight into configuration
‣ Not just netsec/sysops/devops
• More sophistication
‣ LTM… Varnish is an LTM/LB too, but cooler
‣ Caching… Can’t live without it
‣ HTTPhandling...Tune your requests/responses
We needed more.

7. WHERE WE ARE
Traffic Routing, HTTP handling, Caching

8. 8
• Routing by hostname
‣ e.g. status.rackspace.com
• Routing by path
‣ e.g. rackspace.com/knowledge_center
• Environment / ACL handling
‣ Internal URLs
‣ Admin Panels
• Long-term redirection
‣ Old hostnames
‣ Old resources
Local Traffic Management (LTM)

9. 9
• CORS (Cross-Origin Resource Sharing)
‣ Global policy with overrides from backends
• HTTP methods
‣ e.g. preventTRACE method
• Normalization
‣ Fix wacky requests
‣ e.g. rackspace.com///////////cloud
• Framebusting
‣ Global policy
‣ Hostname-specific exceptions (e.g. VWO heatmap tool)
• ESI
‣ When and where appropriate
HTTP Handling

10. 10
• 90%+ hit rate
‣ e.g rackspace.com/cloud?foo=bar -> rackspace.com/cloud
‣ Path-specific overrides
‣ Disregard query params by default
‣ Internal overrides only
• Grace
‣ Invaluable during downtime
‣ Great during maintenance periods / updates
Caching

11. ALL TESTED
Very satisfying, gains confidence.

12. HOW WE GOT HERE
Automation, Continuous Integration, SDLC

13. 13
• Puppet
‣ Configuration automation / simplification
‣ ERB templates
• Jenkins CI
‣ Puppet + Docker = great for idempotent behavior
‣ Run/lint Puppet-generated VCL
‣ Run Varnish tests against latest VCLin Docker container
• Ant
‣ TaskAutomation
‣ Defined set of tasks, no checklists for humans
Automation, automation, automation

14. 14
• Local
‣ Vagrant VMs use same Puppet configs as staging/production
• Docker
‣ Temporary environments for CI
‣ Simplified CI architecture for all applications
• Full Staging Environment
‣ Comprehensive replica of production, top to bottom
‣ Smaller than production, still uses HAtechniques (multiple machines)
• Active/Passive Production
‣ Call it pre-prod
‣ Multiple production envs = better disaster recovery (DR)
Environments

15. 15
• One repository to rule them all
• Better than just documentation
• Developers can contribute
‣ Path for learning VCL
‣ Foundation in varnistest
• DevOps engineers can contribute
‣ Path for learning VCL
‣ Foundation in varnistest
• Peer Review
‣ At least two peers required to sign-off
‣ Preferrably 1 DevOps, 1 Developer
SDLC (Software Development Life Cycle)

16. EVERYBODY WINS
No one is alone, nothing is lost.

17. CLOSING REMARKS
To inspire VCL users to embrace software development practices for all to benefit.

18. THANK YOUContact me – david.porter@rackspace.com
See me afterward for show and tell