AWS Lambda Scaling: Minimizing Latency, Maximizing Throughput & Reducing Costs
•
0 likes•15 views
Talk: "AWS Lambda @ Scale: Designing for High Load" Event: Nerdearla 2023 (https://nerdear.la) Author: Alexis Cuadrado In the realm of serverless computing, AWS Lambda stands as a powerhouse, providing the backend for many of today's most scalable and robust applications. A fully managed compute service capable of running your code at any scale, its surface-level simplicity can be misleading: when operating under a high influx of requests, an adecquate management of the intricate dynamics of the Lambda concurrency model is key to success. In this talk, we will demystify Lambda concurrency and throughput to help you design your applications for optimal performance under high load. We will explore strategies for optimizing individual Lambda functions for latency and illustrate how to strategically employ concurrency controls. Tailored for AWS developers, architects, and system administrators, this session seeks to transform AWS Lambda from a 'black box' to a transparent and fine-tuned engine for your serverless applications.
Recommended
Recommended
More Related Content
Similar to AWS Lambda Scaling: Minimizing Latency, Maximizing Throughput & Reducing Costs
Similar to AWS Lambda Scaling: Minimizing Latency, Maximizing Throughput & Reducing Costs (20)
Recently uploaded
Recently uploaded (20)
AWS Lambda Scaling: Minimizing Latency, Maximizing Throughput & Reducing Costs
- 1. BY ALEXIS CUADRADO AWSLAMBDA@SCALE DESIGNING FOR HIGH LOAD
- 2. LAMBDA @ SCALE #NERDEARLA LOOK, IT SCALES AUTOMATICALLY! MANAGED SERVICE ACCOUNT CONCURRENCY COLD STARTS THROTTLING BURST CONCURRENCY EXECUTION COSTS
- 4. THREEPERFORMANCEINDICATORS $ LATENCY THROUGHPUT COSTPER TRANSACTION seconds or milliseconds transactions per second (TPS) USD MINIMIZE MAXIMIZE MINIMIZE TIP: Define proper Service Level Objectives (SLOs) LAMBDA @ SCALE #NERDEARLA (meet demand)
- 5. MINIMIZING LATENCY BY ALEXIS CUADRADO AWSLAMBDA@SCALE MAKE EVERY MILLISECOND COUNT
- 6. ANATOMYOFALAMBDAINVOCATION 1 2 RUNTIME 3 COMPUTE SUBSTRATE EXECUTION ENVIRONMENT HANDLER CODE STATIC CODE 4 5 Download Code Start Environment Bootstrap Runtime Run Static Code Run Handler 1 2 3 4 5 LAMBDA @ SCALE #NERDEARLA
- 7. initializing executing available invocation LONGLIVETHEEXECUTIONENVIRONMENT! 1 2 3 TIME EXECUTION ENVIRONMENT LAMBDA @ SCALE #NERDEARLA
- 8. STARTEDONTHECOLDFOOT initializing executing available invocation 1 2 3 E1 E2 4 5 cold start Download Code Start Environment Set up Runtime Run Static Code Run Handler 1 2 3 4 5 LAMBDA @ SCALE #NERDEARLA
- 9. DISSECTING LATENCY Download Code Start Environment Bootstrap Runtime Run Static Code Run Handler 1 2 3 4 5 COLD START ( INITIALIZATION ) EXECUTION LAMBDA @ SCALE #NERDEARLA
- 10. OPTIMIZEHANDLERLOGIC EMPLOY EFFICIENT ALGORITHMS A B C Put those hard-won whiteboarding skills to use Parallelize I/O operations (e.g. S3 downloads) Use Step Functions LAMBDA @ SCALE #NERDEARLA AVOID ORCHESTRATION MOVE WORK OUTSIDE HANDLER D MULTI-THREADING Download Code Start Environment Set up Runtime Run Static Code Run Handler 1 2 3 4 5 Reusable objects should be statically initialized
- 11. TURNUPTHERAM! Example Python Function: Return all prime numbers between 0 and 10K MEMORY (MB) EXECUTION DURATION (MS) 128 170 256 80 512 40 1024 20 1536 17 3008 17 MEMORY = vCPU NETWORK THROUGHPUT lowest latency negative returns beyond this point Beware of LAMBDA @ SCALE #NERDEARLA Download Code Start Environment Set up Runtime Run Static Code Run Handler 1 2 3 4 5 TIP: Enable Lambda Insights for profiling
- 12. STATICINITIALIZATIONDONERIGHT + 120 ms LAMBDA @ SCALE #NERDEARLA Download Code Start Environment Set up Runtime Run Static Code Run Handler 1 2 3 4 5
- 13. PACKAGESIZEMATTERS Download Deployment Package Start Environment Bootstrap Runtime Download Deployment Package Start Environment Bootstrap Runtime Run Static Code Run Handler 50 MB 20 MB TIPS: • Audit and remove unused dependencies • Use minifiers (e.g. node-minify) LAMBDA @ SCALE #NERDEARLA 1 1 3 4 5 2 3 2
- 14. ONCOLDSTARTFREQUENCY LAMBDA @ SCALE #NERDEARLA ? ? ? ? COLD WARM Isn’t that what we’re all asking in our own lives? HOW CAN WE GET MORE OF THIS? ... AND LESS OF THIS?
- 15. CONCURRENCY t₀ t₁ E3 concurrency = 1 t₂ E1 E2 concurrency = 3 1 2 3 4 TIME LAMBDA @ SCALE #NERDEARLA
- 16. PROVISIONALLTHECONCURRENCY! t₀ E3 E1 E2 provisioned concurrency = 3 1 2 3 4 5 t₁ concurrency is provisioned requests start arriving TIME NO COLD STARTS! LAMBDA @ SCALE #NERDEARLA
- 17. AUTOSCALING TIME DEMAND / CONCURRENCY LAMBDA @ SCALE #NERDEARLA
- 18. MAXIMIZING BY ALEXIS CUADRADO AWSLAMBDA@SCALE THROUGHPUT LEAVE NO REQUEST BEHIND
- 19. AFUNCTION’STHROUGHPUT 1 2 1 2 1 2 3 4 = 2 TPS THROUGHPUT = CONCURRENCY LATENCY = 4 TPS = 4 TPS = 500 ms = 1 = 500 ms = 500 ms = 500 ms = 2 = 250 ms = 250 ms = 250 ms = 250 ms = 1 3 4 LAMBDA @ SCALE #NERDEARLA
- 20. REDUCINGLATENCYTOGAINTHROUGHPUT 250 ms 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 9 10 1 2 4 3 = 4 TPS 1 sec 100 ms 50 ms 125 ms 1 2 3 4 = 8 TPS = 10 TPS = 1 LAMBDA @ SCALE #NERDEARLA ?
- 21. OH,THERATELIMITS 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 100 ms 50 ms 50 ms 4A 3 4B = 20 TPS = 10 TPS = STILL 10 TPS = 1 Throughput capped due to rate limiting Additional invocations will increase concurrency or be throttled LAMBDA @ SCALE #NERDEARLA 1 3 5 7 9 11 13 15 17 19 2 4 6 8 10 12 14 16 18 20 ASYNCHRONOUS INVOCATIONS SYNCHRONOUS INVOCATIONS No limit!
- 22. ACCOUNTCONCURRENCY TIME E1 E100 E1 E800 THROTTLED X concurrency = 200 concurrency = 800 account concurrency = 1000 ANOTHER FUNCTION OUR FUNCTION LAMBDA @ SCALE #NERDEARLA Everyone gets their share
- 23. RESERVEDCONCURRENCY TIME E1 E300 E1 E100 THROTTLED X concurrency = 300 concurrency = 100 account concurrency = 1000 OUR FUNCTION ANOTHER FUNCTION reserved concurrency = 900 -1900 LAMBDA @ SCALE #NERDEARLA
- 24. BURSTCONCURRENCY TIME (MINUTES) CONCURRENCY (UNITS) 5K 3000 2500 1000 1500 1000 0 0 500 1000 2500 1 0 3 2 4 6 5 7 9 8 2K 1K 3K 4K +500 +500 +500 +500 +0 +0 +0 +500 +500 BURST QUOTA = 3000 AVAILABLE BURST ACCOUNT CONCURRENCY = 5000 BURST THROTTLE ZONE ACCOUNT THROTTLE ZONE BURST THROTTLE ZONE FUNCTION CONCURRENCY Super (OR HOW FAST CONCURRENCY CAN RISE) LAMBDA @ SCALE #NERDEARLA
- 25. MINIMIZING BY ALEXIS CUADRADO AWSLAMBDA@SCALE $ COSTS GET THE BIGGEST BANG FOR YOUR BUCK
- 26. HOWLAMBDAPRICINGWORKS $ COSTPER TRANSACTION also COST PER EXECUTION $ = COMPUTE CHARGES REQUEST CHARGES Rates vary based on Region and CPU Architecture Free Tier available Elegible for Savings Plans fixed fee per request EXECUTION DURATION ALLOCATED MEMORY LATENCY determines influences LAMBDA @ SCALE #NERDEARLA
- 27. POWERTUNING alexcasalboni/aws-lambda-power-tuning OPTIMAL MEMORY CONFIGURATION LAMBDA @ SCALE #NERDEARLA
- 28. ARMYOURSELF(WITHGRAVITON2) COST LATENCY LAMBDA @ SCALE #NERDEARLA
- 29. PUTTINGITALLTOGETHER BY ALEXIS CUADRADO AWSLAMBDA@SCALE BEFORE OUR BRAINS BURST
- 30. WHATWE(HOPEFULLY)LEARNEDTODAY KEY PERFORMANCE METRICS Getting our Lambda Functions to perform as required THE CHALLENGE OF SCALE COLD STARTS HOW LAMBDA PROCESSES INVOCATIONS Execution Environments CONCURRENCY HOW LAMBDA SCALES HOW PRICING WORKS QUOTAS CONTROLS ACCOUNT CONCURRENCY BURST CONCURRENCY PROVISIONED CONCURRENCY RESERVED CONCURRENCY (how fast) (how much) (pre-warm) (set aside) POWER TUNING $ LATENCY - THROUGHPUT - COST PER TRANSACTION WHAT FACTORS AFFECT LATENCY THROUGHPUT CONSIDERATIONS Interplay between memory, execution time and costs COLD START EXECUTION Rate Limits, Concurrency Quotas and Controls Relation to Latency and Concurrency OPTIMIZATION TECHNIQUES Price-performance gains with arm64 (Graviton2) LAMBDA @ SCALE #NERDEARLA
- 31. LAMBDA @ SCALE #NERDEARLA THANKYOU @alexscuadrado /in/alexis-cuadrado FOLLOW ME 𝕏 alexis.hashnode.dev