Reach me on twitter - @matt_billock
Covers some of the lessons learned by Backand as we worked with AWS Lambda over a year in our multi-tenant serverless development platform
4. AWS Lambda
● Created in 2014, launched at
re:Invent
● Kicked off "serverless" (caveat)
● Function-as-a-Service
○Deploy code, not applications
5. AWS Lambda Details
● Supports Node.JS, Python, C#, and
Java
● Can call to server functionality
● Code deployment options
○ Uploading as a ZIP file (S3 or HTTP)
○ In-browser editor
15. ● Scraped logs of 2.6 million function
calls
● Pulled from CloudWatch on AWS
○ Function execution only
● Data on computing resources
consumed only
Examining Lambda Performance
16. ● Graphs built on a sample of full data set
(26,000 vs 2.6 million)
○ Sample is not random
● No relation to size of Lambda function
● Confounding factors may be present
● Significant outliers
First, Caveats
18. Memory vs Execution Time
● Does not incorporate HTTP Request
● Not a very strong correlation (r=-0.03)
● Three distinct groupings
● Tip: More memory allocated = faster
processor
20. Runtime Histogram
● Many functions run in < 100 ms
● Ignores code characteristics
● Ignores significant outliers on both ends
● Bimodal due to function characteristics
21. Improving the Analysis
● Capture full cycle times if possible
● Correlate runtimes with size of function
● Algorithmic complexity analysis
● Randomize sample selection
23. Four problems to solve:
● Problem 1 - Lambda Ownership
● Problem 2 - Deduplication
● Problem 3 - Triggering Lambda Functions
● Problem 4 - User Security
Lambda Integration
24. Problem 1 - Lambda Ownership
Option 1: User ownership
● Users host Lambda functions in their AWS
account
● Backand uses provided credentials to call
Lambda
● Maximizes freedom at the expense of ease of
use (also, security)
25. Problem 1 - Lambda Ownership
Option 2: Backand ownership
● Backand hosts Lambda functions in internal
AWS account
● No need for user's AWS credentials
● Maximizes ease of use at expense of user
freedom
Solution selected: Backand owns code
26. ● Lambda function names need to be unique
● Name length limits:
○ 140 characters for full ARN
○ 64 character subset for function name
Solution Selected: prepend guaranteed unique data
● Backand app names have a uniqueness constraint
● Could also use GUID, but this reduces name length
to 32 characters
Problem 2 - Deduplication
27. ● Most triggers AWS-based
○ S3, DynamoDB, Cloudwatch
○ None work for our use case
● API Gateway
○ Positive: Allows HTTP trigger
○ Negative: Complex configuration
○ Negative: Return value requirements
Solution selected: AWS SDK
Problem 3 - Triggering Lambda
28. ● Default: configure IAM profile and users
○ Need to give every dev access to AWS
○ Does not integrate with org security
○ Needs to be managed in multiple places
● Clients want flexibility and security
○ Need Single Sign-On
○ Need custom third-party security tools
○ Need ease of management
Solution selected: Lambda Launcher
Problem 4 - User Security
30. Debugging Lambda Functions
● Cloudwatch Integration
○ Provides call record, and console.log()
● Custom parameter input
○ Modal dialog presented before each run
● … and that's it
32. Emulating trigger event
● Each trigger sends different data
● Dump data from a sample call
(console.log())
● Keep in mind any transformations (API
Gateway)
33. Lambda Machine Environment
● Based on Amazon Machine Instance
● Dynamically provisioned (hot vs cold)
● Temporary storage (/tmp, 500 MB)
○ NOT guaranteed to persist
● Cannot accurately model machine
34. Invoking Lambda
● External Function Call (entry point)
● Custom parameters in "event" object
● Execution context details in "context"
● Callbacks (if supported) in "callback"
35. Debugging Lambda Locally
● Execution environment hard to
emulate
● Need to mimic input format from
action
● Need to mimic output format
● Need a test harness
36. Creating a Test Harness (Node.js)
● Function Prototype
● Including Handler
● Mimicking Lambda Invocation
○ Parameters in event object
○ Handlers in the context object
38. Test Harness Notes
● Callback mechanism
○ Need to adapt for each language
● Using callbacks in context parameter
○ Not supported, overloads context object
● Not a perfect substitute
○ Calls the function, but not from the same
environment
39. What do we get?
● Per-line output via console.log
● Immediate feedback from command
● Breakpoints and other local debugging
tools
● Unit and Integration tests
● CI/CD compatible (with CLI)
40. Deploying Your Code
● Zip-file from S3, or manual upload
○ Gotcha: zip only the source
code, not the parent folder!
● Test upload
● Publish new API
● Complex, non-intuitive
42. Lambda through a CLI
● Initialize a function
○ backand function init
● Copy your Node.JS code into created
directory and iterate
● Deploy
○ backand function deploy
43. Calling Your Lambda
● Authentication
○ can be anonymous
○ api.backand.com/1/token
● cURL
curl -H "<auth header>" "https://api.backand.com/1/function/general/<name>"
● JavaScript (with Backand SDK)
backand.function.get(...)
44. What is Backand?
● Serverless app platform
● Manages your app's database
● Manages your app's security and
authentication
● Provides custom server-side code execution
● Provides hosting options
● Provides logging, analytics, and more
45. Backand Features
● Automated REST API
● Bring your own Database
● Custom JavaScript Actions
● Server-Side Code Execution
● Batch and Bulk Processing
● Scheduled Tasks
● Automated messages
● Real-time Communications
● User and Role-based
Security
● Social Media Authentication
● GUI Schema Editor
● Custom Queries
● REST API Playground
● Sample Code Generation
● Hosting
● Detailed Analytics
● Logging
● Multi-Platform SDK
● Single Sign On Support
46. ● Live online demo every other Wednesday
● Webinar - Invoking Lambda through Alexa
Thursday, June 22nd, 11 AM Pacific
Learn more at https://www.backand.com
Want to know more?