From launching Dashbird one and a half years ago, our team has been in contact with a lot of companies building serverless applications and one of the problem areas that keeps poppping up are performance or cost issues. Performance problems usually present themselves when some best practices are overlooked or a suboptimal architecture decision have been made. This article is focused on how to look for optimisation opportunities and to capitalise on those opportunities.
While serverless applications usually consist of event sources, lambdas functions and external services, functions almost always present the most opportunity for optimisation. Mainly because as a developer, you have less control over (managed) event sources and external services. To spot opportunities for efficiency, lets look under the hood for a second and identify where we should look.
Anatomy of a Lambda function
Breaking a Lambda function into pieces you can see there are multiple places to look for optimisations.
- Your function code is usually the biggest opportunity for optimisation and a threat of making mistakes.
- Language runtime, Python, Go binary Node.js etc. does not allow optimisation.
- Execution environment is a somewhat unintuitive place to think about but it can have an effect on performance. For example, how an function is invoked, is it in VPC and needs an ENI? Are there too many redundant packages?
- Compute substrate also allows no optimisation. AWS controls this and there’s nothing you can do about this.
Structuring Lambda code for performance
Let’s start by what you can do in your code since it’s by far the most significant part of your application. Let’s break it down into smaller parts first…
- event object from the event source
- context object provides methods to interact with runtime info (request ID, log group, timeout, etc.)
- initialisation inclusion of dependencies, establishing database connections and importing secrets and initialisation variables.
- handler function the function to be executed upon each invocation
- business logic outside of the handler function
Since initialisations takes place once for every time a container starts
For function initialisation, the key things to think about is reducing the size of dependencies and reusing database connections.
- minimize dependencies
- use pre-handler logic strategically
- Share secrets based on application scope
- single fn: env vars are most lightweight but dont scale
- multi fn: parameter store
- think about how re-use affects variables, connections and dependency usage
Concise function logic
- Separate Lambda handler code from core logic
- Use functions to TRANSFORM, not TRANSPORT
- Dynamic logic via configuration - if it does something differently in dev that in prod, don’t build that into your code but use env variables.
- Read only what you need - efficient requests against DB, use views and S3 select etc.
- No orchestration in code
- retries of network requests etc
- failure handling INSTEAD: use step functions (retry logic, orchestration logic, workflow logic out of lambda)
- Project & repository scoping (kk)
mostly you wont interact with it, but you can look where the logs go and what the timeout is and can make decisions wether to but something in the queue or execute it right now.
Lambda processes a single event per execution environment and reuses containers for consequtive events. The container is kept alive for 5-15 minutes and amount of concurrent requests dictates the number of “frozen” containers that AWS keeps alive. For consequtive requests, only the handler function is re-executed and not the initialisation part.
Optimising execution environment
- More memory == More CPU and I/O (proportionally)
- Use AWS X-ray to profile your workload
- >1.8GB === 2 cores, but you might not use/need it
- Think deeply about your execution model and invocation source needs (not everything needs to be an API)
- Understand various aspects of queues, topics and streams when using them
- VPC has benefits but isn’t necessary for security
- Minimize scope for IAM permissions
This article is based on the talk about optimizing serverless applications @reInvent 2018