Start free trial

Central data platform for your serverless environment.

Get full access to all premium features for 14 days. No code changes and no credit card required.

Password: 8+ characters, at least one upper case letter, one lower case letter, and one numeric digit

By signing up, you agree to our Privacy policy and
Terms and Conditions.

Cold Start Impact On Latency

Share

Even in this field of work, not everything can be perfected 100%. There are always some situations and cases that will force you to go back or even remain in the present spot, despite your wish to keep going forward at your own pace. In this article, we’ll talk about the cold start impact on latency. What is it? How to fight against it? Is there a successful way of avoiding it or not? Follow up this link to our website and find many useful answers regarding the serverless technology and its benefits.

What Is A Cold Start And Why Do We Care?

Going serverless has a tremendous amount of benefits as well as it has its downfalls. In some cases, it’s not the cheapest way of hosting. But again, in other cases, it simplifies and quickens the turnaround time which results with much more free time for other things to be done. All serverless computing has the same weak spot known as the cold start. Even the AWS Lambda is no exception to this dreadful fact. Let’s roll-back and explain why is the cold start so meaningful and why do we care about it. The serverless cold start is happening when your code is executed for the first time by your cloud provider. Your code requires to be downloaded, placed in a container, to be booted, and of course, it needs to be primed to run. All of these requirements add up to 1.5s of latency. The bright side of it all is that these cold starts are the outliers that affect only about 5% of executions. Therefore, it’s important to mention that they do not happen all the time, but still are an essential aspect to think about when you’re creating your application.

Cold Start Latency

The cold start latency depends on numerous variables like the language you’re using, how many libraries you’re using, or how much code do you have. Also, it depends on the configuration of the Lambda function environment as well as if you need to connect to Virtual Private Cloud (VPC) resources, https calls, code size, memory size, and so on. A lot of these aspects are under control by the developer which means that it’s frequently possible to reduce the startup latency which happened by a cold start. Should we be worried about the cold start? The answer to that question really depends on the style and traffic shape of your app. For example, if you’re writing a low-latency trading app, you most probably wouldn’t want to use the cloud-hosted FaaS systems, regardless of the languages used for its implementation.

Cold Start Optimization

In the raging battle against the cold start and latency, we’ve come across a couple of ways to optimize the cold starts. The two tactics to battle against the cold starts are:

1. Minimize the cold start duration

In simple words, this means cutting down the latency of the cold start itself. To achieve the lowering of the time impact of cold starts, basically, you’d need to write functions via interpreted languages. Cold start latency is under 1s with Python and Node.js. Language like Go is yet another example of low cold start latency. Choosing a higher memory setting for your functions might also do the trick, but it also provides your functions with much more CPU power. Another way to avoid the latency is to exclude the utilization of VPC since VPC must create ENIs which goes well over 10s upon initialization. Avoid placing your functions in VPC, and it will help you a great deal.

2. Minimize the cold start rate of occurrence

Warming up the function can reduce the occurring cold starts. Sending the pre-scheduled ping events to your function will keep them idle and alive, which means they’re ready to serve requests. Yet again, this brings us to another question: How to handle simultaneous cold starts? There are several modules and plugins which will solve this for you. For Node.js developers, there’s a Lambda Warmer plugin which will help you out with warming up simultaneous functions, and it also allows you to choose the levels you want. It’s compatible with both AWS SAM and the Serverless Framework. It is worth mentioning that the Serverless Framework has another plugin “Serverless WARM-Up Plugin” but note that it doesn’t support the warming of simultaneous functions.

To warm up your functions correctly, see this guideline:

  • Never invoke your function more frequently than once every five minutes;
  • Amazon CloudWatch Event is where you must summon your function;
  • When you run the warming process you should pass-in a test payload;
  • Create a handler logic that even though it’s running the warming, it doesn’t run all function logic.

By utilizing this warmer, you are able to safely keep as many AWS instances running as you wish, which includes the copies of the exact same function in case you expect the concurrent requests. This just might mean that the cold starts are history.

The Conclusion

Unfortunately, there is no available firm solution, and you must accept that there is no guarantee that you won’t experience the cold starts. The ultimate solution must and will come from the cloud provider. You must be prepared for 1-3 seconds cold starts even for the small functions, but know that all cloud providers are aware of this problem and that they’re actively doing their best to optimize the cold start experience which furthermore means fixing the latency issues. If nothing from this article helped with your cold-start/latency issues, try finding a solution again in a few months since FaaS vendors are continually improving their services.

Made by Developers for Developers

Our history and present are deeply rooted in building large-scale cloud applications on state-of-the-art technology. Dashbird was born out of our own need for an enhanced serverless debugging and monitoring tool, and we take pride in being developers.

10,000+ developers trust Dashbird

Dashbird helped us reduce the time to discovery from 2-4 hours to a matter of seconds. This means we’re saving dozens of hours in developer time, which not only can be funneled into improving the product but also means hundreds of dollars are saved every month.

Great onboarding: it takes just a couple of minutes to connect an AWS account to an organization in Dashbird. The UI is clean and gives a good overview of what is happening with the Lambdas and API Gateways in the account.

Dashbird provides an easier interface to monitor and debug problems with our Lambdas. Relevant logs are simple to find and view. Dashbird’s support has been good, and they take product suggestions with grace.

Great UI. Easy to navigate through CloudWatch logs. Simple setup.

Dashbird helped us refine the size of our Lambdas, resulting in significantly reduced costs. We have Dashbird alert us in seconds via email when any of our functions behaves abnormally. Their app immediately makes the cause and severity of errors obvious.

Read our blog

AWS Well Architected Framework in Serverless: Cost Optimization

In this post, we’ll talk about the Cost Optimization (COST) pillar of the Well-Architected Framework (WAF) and what you should focus on in order to build cost-effective serverless environments.

How to Build, Deploy, and Debug a Food Delivery App on AWS

We’re getting down and dirty in this hands-on tutorial on how to build and deploy an event-driven Lambda backed food delivery app, and how to monitor it without using AWS products.

AWS Step Functions Error Handling

In this article, you’ll learn the common reasons behind AWS Step Functions errors and how to assess and handle them.

AWS Well-Architected Framework in Serverless: Reliability Pillar

In this article, we will focus on the AWS WAF Reliability (REL) pillar: the Foundations, Failure and Change Management.

AWS Well-Architected Framework in Serverless: Operational Excellence

This article will discuss the second most crucial pillar of the AWS Well-Architected Framework: Operational Excellence (OPS).

Go to blog