Lambda Metrics That You Should Be Monitoring

What are the crucial AWS Lambda metrics you should definitely be monitoring?

Your application does not need to be “huge” for it to have enough functions and abstraction to get lost in it. As a DevOps engineer, you can’t cover every single factor. Showing relevant facts and asking the right questions is crucial! So when there’s a fire, you can troubleshoot in no time.

Every organization is unique, and every workload has its own utility. That said, we can still have a generalized approach and start by listing a few desirable qualities that you may want from your AWS application:

  • Performance
  • Responsiveness
  • Cost-effectiveness

AWS Lambda Pricing

Lambda pricing is very straightforward, and the billable factors include:

  • Number of requests
  • Compute time
  • Amount of memory provisioned

Compute time and memory provision are coupled. We’ll mention this in more detail further below. Let’s start with the number of requests. The first one million requests are free every month. After that, you will be charged $0.20 per million requests for the remainder of that month. That’s stupid cheap.

In addition to the number of requests, you also pay for the memory allocated to your function along with the computing time that the function takes to do its job. You specify the memory allocation when you deploy a function. The computing time can vary from one invocation to the next but is limited by the maximum timeout which you configured.

Suppose your function runs for two seconds (2000 milliseconds) and has been allocated 512 MB of memory, then you will be billed for 2 * 512 MB = 1 GB-second of utilization. This is called GB-seconds billing, where you pay for the compute time-based on the memory allocation and time your function runs. If your function has 1 GB of memory allocated and runs for one second (1,000 milliseconds), you pay 1 GB-second.

The first 400,000 GB-seconds are free every month. After the free quota is reached, you pay $0.0000166667 per GB-second (not worth scratching your head over, use our cost calculator instead) for the rest of that month. On top of this, you may get some additional charges for resources like an AWS S3 bucket, VPC, DynamoDB, etc.

Amazon’s Pay-for-what-you-use business model does not rely on your needs but the success of your business. If your apps are accessed more often, your organization benefits more, along with a slightly greater AWS bill. This, in turn, benefits Amazon.

AWS Lambda monitoring

Which Metrics Does Lambda Track by Default?

The Lambda service comes with seven metrics for your functions out of the box. Invocations, duration, error count, throttles, async delivery failures, iterator age, and concurrent executions.

Setting them up to alert you when it’s needed is a challenge we can easily solve. Let’s see what these metrics mean in practice.

The invocations metric tells you how often a function was called in a specific time period.

The duration metric tells you about the runtime of your function for a specific time period. It splits them up into the slowest, fastest, and average duration.

The error count is about how often your function crashed in a specific time period. It also includes a success rate in percentage because a hundred errors with a success rate of 99% do mean something different than hundred errors with a success rate of 1%.

The throttled metric shows how many function invocations couldn’t be handled because your function was already scaled up to the concurrency limit of Lambda. The default is 1,000 concurrent executions, but it can be increased if needed.

Async delivery failures occur when your function tries to write to a destination or dead-letter queue but fails.

The iterator age metric is helpful when your function reads from a stream (i.e., Kinesis or Kafka). Such streams are often used, so upstream services aren’t overwhelmed by Lambda’s ability to scale really fast. If your consuming Lambda function is too slow, it can be that the data stays longer and longer in the stream, and thus the iterator age metric rises.

The concurrent execution metric checks how many parallel containers for your functions are loaded in a specific time period. If you get near the region limit of 1,000, you should try either to improve the performance of your functions or talk to AWS about increasing that limit for your account.

Enter Lambda Cold Starts

Each Lambda function runs inside a Docker container. When it’s invoked for the first time, the Lambda service first spins up a new container and then executes the function inside it. This increases latency and may make your application seem slow to the user initially. After this initial latency, the function is kept loaded for a period of time. During that period, new invocations don’t suffer similar latencies and feel much more responsive to the client.

Think of it as accessing files within a high-speed cache; when the files are not used for long periods of time, they’re flushed out of the cache. When you request those files again, it takes longer to open them.

This process is called a cold start and it can be solved in various ways.

Dashbird introduced the cold start monitoring to show you which invocations were cold starts.

cold starts view

The period for which the function stays warm is speculated to be between 5 and 30 minutes, but Amazon has not confirmed anything. If the function relies on ENI to talk to other services, then that adds another layer of latency.

There’s one additional complication, and that’s the concurrency issue. If you receive a burst of traffic simultaneously, AWS scales the function by spinning up more containers to handle all the new requests. This causes a whole different sequence of cold starts which has nothing to do with left idle resources for too long.

Optimizing the right resource

Okay, so we have listed quite a few problems, now let’s solve a few of them. We hope that combining the following strategies can help you achieve a desirable balance between responsive applications and lower bills. After all, if your system becomes sluggish, nobody will use it even if it is cheap.

1. Increasing Memory Allocation

Increasing the memory allocation is accompanied by an implicit increase in CPU allocation. This may very well result in faster execution of your function. Reducing actual execution time to accommodate the latencies caused by cold starts can directly improve the user experience. Moreover, this is the easiest hypothesis to test in higher latencies—the first line of defense for your DevOps team.

AWS allocates more CPU power, if you allocate more memory to your Lambda function.

2. Keeping Functions Warm

Another way to tackle the latency issue is to buy provisioned concurrency for your Lambda functions. Provisioned concurrency asks the Lambda service to keep a specified number of containers with your function warm, so they can respond to events quicker even if they haven’t been called for a long period of time. When the real-world workload comes in, hundreds of tiny little Lambda instances are prepared for the mighty battle.

Dashbird can help you see patterns hiding in users’ behavior. Use it to time your warm up calls. Making numerous requests is often cheaper than increasing memory.

3. Optimize your code

Possibly the cruelest option would be to ask your developers to optimize their code. This doesn’t necessarily involve making changes in the actual logic or functionality, but one can often trim down the dependencies and make the function a bit more lightweight.

It’s also a good idea to keep as much work outside your function’s body as possible. For example, setting up API clients or calculating fixed values from environment variables. This way, the work only slows down the cold start, but not the subsequent calls to your function.

If possible, you should stick to languages like Node.js and Python since they have significantly lower cold start times than their C# or Java counterparts.

You can also try out custom runtimes. If you have very computation-heavy tasks, using a Rust runtime and implementing your function in Rust can save you quite some money. Especially since Lambda functions are billed per millisecond, functions with a high invocation frequency can profit tremendously from such a refactor.

Wrapping up

To make any of the above inferences plausible, you need to know the specific pathology of your misbehaving and cost-inefficient function. This is then followed by asking the right questions, making an educated guess about the solution, then optimizing AWS Lambdas.

Dashbird helps you every step of the way, from keeping track of subtleties like cold starts to knowing whether or not a new solution has made any difference. Of course, there are more parameters to consider, like concurrency and synchrony. They would need a much deeper dive. More about that in one of my next articles!

Stay tuned, and sign up to monitor everything we have went over in this article.


Further reading:

Quick ways to cut cost on your AWS Lambda

Dashbird’s Serverless Events Library: Lambda error messages and debugging

Navigating CloudWatch logs with Dashbird

Using Observability to Scale AWS Lambda [Webinar]

Read our blog

Top 6 AWS Lambda Monitoring Tools

Monitoring helps you identify any performance issues, and it can also send you alerts and notify you of anything you might need to know. This article will cover the top 6 AWS Lambda monitoring tools and explain how they work and are used.

8 Common Mistakes When Using AWS ECS to Manage Containers

In this article, we’ll discuss the potential pitfalls that we came across when configuring ECS task definitions. While considering this AWS-specific container management platform, we’ll also examine some general best practices for working with containers in production.

AWS Lambda Cost Optimization Strategies That Work

From caching Lambda responses to building smaller functions to choosing the right memory configs. In this article, we’re exploring six specific steps you can take to optimize your AWS Lambda costs.

More articles

Made by developers for developers

Dashbird was born out of our own need for an enhanced serverless debugging and monitoring tool, and we take pride in being developers.

What our customers say

Dashbird gives us a simple and easy to use tool to have peace of mind and know that all of our Serverless functions are running correctly. We are instantly aware now if there’s a problem. We love the fact that we have enough information in the Slack notification itself to take appropriate action immediately and know exactly where the issue occurred.

Thanks to Dashbird the time to discover the occurrence of an issue reduced from 2-4 hours to a matter of seconds or minutes. It also means that hundreds of dollars are saved every month.

Great onboarding: it takes just a couple of minutes to connect an AWS account to an organization in Dashbird. The UI is clean and gives a good overview of what is happening with the Lambdas and API Gateways in the account.

I mean, it is just extremely time-saving. It’s so efficient! I don’t think it’s an exaggeration or dramatic to say that Dashbird has been a lifesaver for us.

Dashbird provides an easier interface to monitor and debug problems with our Lambdas. Relevant logs are simple to find and view. Dashbird’s support has been good, and they take product suggestions with grace.

Great UI. Easy to navigate through CloudWatch logs. Simple setup.

Dashbird helped us refine the size of our Lambdas, resulting in significantly reduced costs. We have Dashbird alert us in seconds via email when any of our functions behaves abnormally. Their app immediately makes the cause and severity of errors obvious.