Start free trial

Central data platform for your serverless environment.

Get full access to all premium features for 14 days. No code changes and no credit card required.

Password: 8+ characters, at least one upper case letter, one lower case letter, and one numeric digit

By signing up, you agree to our Privacy policy and
Terms and Conditions.

What is the ideal retention period for application logs

Ready to start monitoring your AWS Lambda application?

Dashbird Banner

Instantly detect and prevent known and unknown serverless errors!

Get free account

What is the Ideal Retention Period for Application Logs?

That is a common question I see among developers. Most of the time, nobody cares about system logs. But when things go south, we absolutely need them. Like water in the desert, sometimes!

At Dashbird, we have a list of criteria compiled to determine a reasonable retention policy for application logs. There is no one-size-fits-all, though. The analytical dimensions below will give a relative notion of how long the retention period should be.

Criticality

Different retention policies for each part of the system can be set depending on how important they are. Mission-critical components can have an extended period to provide extra peace of mind. Services with a marginal value may have logs dumped in two days, for example.

Security

Applications that deal with sensitive or personal data, as well as high-risk processing, should also have an extended retention policy. Services taking care of credit card authorization and user authentication are some examples.

Each system requires its own analysis. A ride-hailing app, for instance, may need to keep trip logs for a certain period to act on any issues reported later. Services that track customer behavior on the app for improvement purposes, on the other hand, might not need a longer retention policy.

Law-Enforcement

Internet services that can be used for illegal activity, such as payment processors or online gaming, would need an extended or even perpetual retention policy to comply with law enforcement.

That could also be true for companies processing personal data under privacy regulations, as they might need to produce evidence of consent.

Accounting and tax compliance software is also likely to be subject to longer retention policies – possibly for years – as government authorities and corporate auditors may be entitled to conduct retroactive analysis.

Maturity

For systems that have been around for a while and are not under constant development of features, new issues will rarely happen. Therefore, in mature software systems, a shorter retention policy might be acceptable to reduce storage costs.

Frequency

Applications that run infrequently (i.e. once every month) should have extended retention policies, as developers might need to verify multiple executions to track down the source of an issue. The low execution frequency would require going back further in time to conduct the debugging.

Cost-effectiveness

Before settling on a retention policy, make sure the cost will be compatible with the project. Project how much data should be generated and how expensive storage is for the intended period. Even though we might want to be safe and store for a long time, it might be reasonable to settle on a more cost-effective option.

Time to Discovery and Resolution

Track how long it usually takes for your development team to discover and resolve an issue. Make sure the log retention policy gives enough room for debugging.

One way to reduce the average time to discovery and resolution of application issues is by employing automated alerting systems and having an efficient way for developers to quickly browse logs and narrow down the root causes of errors.

Read our blog

6 AWS Lambda Cost Optimization Strategies That Work

From caching Lambda responses to building smaller functions to choosing the right memory configs. In this article, we’re exploring six specific steps you can take to optimize your AWS Lambda costs.

How we built a serverless “Stonks” checker API for Wall Street Bets

We built a serverless Hot Stock Checker API that keeps track of trending stocks on Wall Street Bets on Reddit so that you’ll never miss out on the next GME situation. This is how we deployed, tested and monitored the app.

Dashbird becomes Gartner Cool Vendor 2021!

We’re officially cool! Dashbird is extremely proud to be named as a Cool Vendor by Gartner in Monitoring, Observability, and Cloud Operations in their 28 April report on “Cool Vendors in Monitoring, Observability and Cloud Operations”.

More articles

Made by developers for developers

Dashbird was born out of our own need for an enhanced serverless debugging and monitoring tool, and we take pride in being developers.

What our customers say

Thanks to Dashbird the time to discover the occurrence of an issue reduced from 2-4 hours to a matter of seconds or minutes. It also means that hundreds of dollars are saved every month.

Great onboarding: it takes just a couple of minutes to connect an AWS account to an organization in Dashbird. The UI is clean and gives a good overview of what is happening with the Lambdas and API Gateways in the account.

Dashbird provides an easier interface to monitor and debug problems with our Lambdas. Relevant logs are simple to find and view. Dashbird’s support has been good, and they take product suggestions with grace.

Great UI. Easy to navigate through CloudWatch logs. Simple setup.

Dashbird helped us refine the size of our Lambdas, resulting in significantly reduced costs. We have Dashbird alert us in seconds via email when any of our functions behaves abnormally. Their app immediately makes the cause and severity of errors obvious.