From launching Dashbird one and a half years ago, our team has been in contact with a lot of companies building serverless applications and one of the problem areas that keep popping up are performance or cost issues. Performance problems usually present themselves when some best practices are overlooked or a suboptimal architecture decision have been made. This article is focused on how to look for optimization opportunities and to capitalize on those opportunities.
While serverless applications usually consist of event sources, lambdas functions, and external services, functions almost always present the most opportunity for optimization. Mainly because as a developer, you have less control over (managed) event sources and external services. To spot opportunities for efficiency, let’s look under the hood for a second and identify where we should look.
Anatomy of a Lambda function
Breaking a Lambda function into pieces you can see there are multiple places to look for optimizations.
- Your function code is usually the biggest opportunity for optimization and a threat of making mistakes.
- Language runtime, Python, Go binary Node.js etc. does not allow optimization.
- Execution environment is a somewhat unintuitive place to think about but it can have an effect on performance. For example, how a function is invoked, is it in VPC, and needs an ENI? Are there too many redundant packages?
- Compute substrate also allows no optimization. AWS controls this and there’s nothing you can do about this.
Structuring Lambda code for performance
Let’s start with what you can do in your code since it’s by far the most significant part of your application. Let’s break it down into smaller parts first…
- event object from the event source
- context object provides methods to interact with runtime info (request ID, log group, timeout, etc.)
- initialization inclusion of dependencies, establishing database connections, and importing secrets and initialization variables.
- handler function the function to be executed upon each invocation
- business logic outside of the handler function
Since initializations take place once for every time a container starts
For function initialization, the key things to think about are reducing the size of dependencies and reusing database connections.
- minimize dependencies
- use pre-handler logic strategically
- Share secrets based on application scope
- single fn: env vars are most lightweight but don’t scale
- multi fn: parameter store
- think about how re-use affects variables, connections, and dependency usage
Concise function logic
- Separate Lambda handler code from core logic
- Use functions to TRANSFORM, not TRANSPORT
- Dynamic logic via configuration – if it does something differently in dev than in prod, don’t build that into your code but use env variables.
- Read-only what you need – efficient requests against DB, use views and S3 select etc.
- No orchestration in code
- retries of network requests etc
- failure handling INSTEAD: use step functions (retry logic, orchestration logic, workflow logic out of lambda)
- Project & repository scoping (kk)
Mostly you won’t interact with it, but you can look where the logs go and what the timeout is and can make decisions whether to put something in the queue or execute it right now.
Lambda processes a single event per execution environment and reuses containers for consecutive events. The container is kept alive for 5-15 minutes and the amount of concurrent requests dictates the number of “frozen” containers that AWS keeps alive. For consecutive requests, only the handler function is re-executed and not the initialization part.
Optimizing execution environment
- More memory == More CPU and I/O (proportionally)
- Use AWS X-ray to profile your workload
1.8GB === 2 cores, but you might not use/need it
- Think deeply about your execution model and invocation source needs (not everything needs to be an API)
- Understand various aspects of queues, topics, and streams when using them
- VPC has benefits but isn’t necessary for security
- Minimize scope for IAM permissions
This article is based on the talk about optimizing serverless applications @reInvent 2018