Safely Storing Secrets in Git with Blackbox
•
18 likes•14,248 views
A presentation given at PuppetCamp NYC 2014 about why Puppet users should stop storing secrets in Git/Hg and encrypt them instead. TLDR: It enables collaboration.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Safely Storing Secrets in Git with Blackbox
Similar to Safely Storing Secrets in Git with Blackbox (20)
Recently uploaded
Recently uploaded (20)
Safely Storing Secrets in Git with Blackbox
- 1. The BlackBox project Safely storing secrets and credentials in Git/Mg (mostly for use by Puppet) Tom Limoncelli, SRE, StackExchange.com Blog: EverythingSysadmin.com My new book! the-cloud-book.com
- 2. StackExchange.com 125+ Q&A Communities ServerFault.com StackOverflow.com (We <3 Puppet!)
- 3. What are secrets? Anything you don’t want exposed externally. ● SSL Certificates (the private bits) ● Passwords ● API keys
- 6. If you store secrets in git, you’re gonna have a bad time.
- 7. ● Laptops get stolen. ● Workstations have guest accounts ● Git server “Circle of Trust” includes: ○ Everyone with admin access to workstations. ■ Your desktop support people? ○ Everyone with admin access to your git server: ■ Server team, storage team, backup team ○ Everyone you collaborate with that wants read-only access to Puppet manifests.
- 8. You have 3 bad options: 1. Deny git access. (Hurts collaboration) 2. Permit git access. (Hurts security) 3. Email individual files. (Hurts… just hurts)
- 9. Option 4: Encrypt secret parts ● If a file contains secrets, encrypt before checking into Git. ● Need to edit a secret? ○ Decrypt - Edit - Encrypt
- 10. What about Puppet master? ● After “git pull”, decrypt all files. ○ Automate this as part of CI. ● Files are unencrypted “at rest”. ● This does not decrease security: ○ No worse than what we were doing before. ○ If you can break into root or puppet on the master, you’ve already won.
- 11. Easy, right? Decrypt: Encrypt:
- 12. Easy, right? Decrypt: Encrypt: ● ...and don’t make any typos when entering the command ● ...and don't accidentally check in the unencrypted version
- 13. Security is 1% technology plus 99% following the procedures correctly. Any process with more than 1 step probably won't be followed consistently most of the time. Related reading: "Why Johnny Can't Encrypt: A Usability Evaluation of PGP 5.0”, Alma Whitten", Usenix Security 1999
- 14. Therefore…. we automate Introducing: Blackbox Scripts for keeping Puppet secrets in git/hg.
- 15. User commands: Decrypt for editing: Encrypt when done:
- 16. First time a file is encrypted: Enroll a file into the system:
- 17. Commands that act on all GPG files: Decrypt all files: (for use on puppet master) Re-encrypt all files: (after new users added)
- 18. Everyone has their own key This doesn’t use “symmetric encryption” where there is one passphrase to decrypt/encrypt all files. We maintain a keyring of: ● Each person that should have access. ● A key for the Puppet master.
- 19. Indoctrinate a new user: 1. New user does this: ●● ● (Currently a doc, not a script. Patches gladly accepted.)
- 20. Indoctrinate a new user: 2. Existing admin does this:
- 21. Demo: Edit a file
- 22. Demo: Edit a file
- 23. Demo: Edit a file
- 24. Demo: Edit a file
- 25. Demo: Edit a file
- 26. Demo: Edit a file
- 27. Demo: Edit a file
- 30. Code is open source as of 7/2014 ● Entirely written in bash. ● MIT License. ● Download it now: ○ https://github.com/StackExchange/blackbox
- 31. In the project’s first 9 months: StackExchange/ServerFault has eliminated plaintext secrets in our Puppet git repo. ● 7 SREs+Devs sharing the repo securely. ● 50+ files now stored encrypted. ○ Mostly SSL certs and SSH private keys. ● 40+ individual passwords/API keys: ○ Everything from SNMP communities, SaaS API keys, and many many passwords.
- 32. Future plans ❏ Open source scripts. ❏ More usability enhancements. ❏ Better setup documentation.
- 33. Join the open source project http://github.com/StackExchange/blackbox
- 34. Q&A URLs from this talk: https://github.com/StackExchange/blackbox EverythingSysadmin.com
- 35. Shameless plug Pre-order now! Save 35% Ships in September. informit.com/TPOSA Discount code TPOSA35 Read “rough cuts” today: safaribooksonline.com
- 36. Q&A URLs from this talk: https://github.com/StackExchange/blackbox EverythingSysadmin.com the-cloud-book.com informit.com/TPOSA (code TPOSA35)
- 37. Why didn’t we use eyaml? ● Easier transition. No Puppet code changes for big files like SSL certs. ● Faster. Zero run-time performance impact on master. ● eyaml didn’t exist when we started.