Start free trial

Central data platform for your serverless environment.

Get full access to all premium features for 14 days. No code changes and no credit card required.

Password: 8+ characters, at least one upper case letter, one lower case letter, and one numeric digit

By signing up, you agree to our Privacy policy and
Terms and Conditions.

Mike Rahmati: My Serverless journey with Cloud Conformity

Share

Mike Rahmati

Mike Rahmati is the Head of the Advisory Board at Dashbird. He is the Co-Founder and CTO of Cloud Conformity (acquired by Trend Micro) – a Cloud Security Posture Management Solution – one of the largest and earliest adopters of serverless. 

Mike is also an active AWS Community Hero. In this article, he shares his journey and experience with serverless.

Cloud Conformity was founded in 2014 as a result of our own experience of issues migrating to the cloud. It was clear early on that as a startup, resources like DevOps would be limited, which is where utilizing serverless made simple sense – we could offload most activities to AWS allowing us to focus on the business itself

The beginning

The serverless ecosystem was very young back then, and a few mistakes were made and learned from along the way. Most notably that we essentially had one fat Lambda for a while, which later needed redesigning and re-architecting! Today, we have over 1,000 Lambda functions within our architecture, use AWS Step Functions to stitch it all together, AWS API Gateway to expose the backend to customers and use many of the other commonly used AWS Serverless services like DynamoDB, S3, ElasticSearch and KMS. 

It’s been quite the journey for us but we are now one of the largest and earliest adopters of serverless. 

Cloud Conformity and serverless

From my own experience and meeting with many others in the community, I fully believe that serverless is the new normal and it’s the future for most organizations. When competing in business, speed is king whether that’s for new product development or better user engagement, however, of course, this doesn’t mean serverless is at the core of the business itself.

At Cloud Conformity, serverless wasn’t our main business but we heavily relied on it and therefore needed a tool that could help so we could continue to focus on business logic. 

As a Cloud Security Posture Management (CSPM) Solution, Cloud Conformity helps customers: to detect misconfigurations in your cloud architecture providing the rationale behind good and bad practices, to correct the misconfigurations, and prevent them or others from happening again. To produce production-quality code in serverless, considerations for reliability, cost and security must be front of mind and I’d always recommend getting these in place as soon as possible. The fundamentals, such as using AWS X-Ray and the rule of least privilege with IAM, are things people often overlook until it’s too late.

The serverless challenge

We knew we needed to have speed on our side because of market conditions and serverless enabled this with quick builds going from prototype to market in remarkable speed, however, as I’ve learned myself and seen from others, serverless architecture scales much faster than you expect.

The tipping point when serverless starts to work at scale is much more about how the services are running, as opposed to the number of components. For example, a bank in Australia uses serverless as part of their Amazon Alexa service, giving customers the opportunity to request details of their bank account. The service uses a Lambda function that could be invoked thousands of times a day, prompting the need for scalability and the need to know if it ever failed

I like to see serverless like Lego; it’s important to have a solid foundation with correct configurations from the start. Having this strong base means that when, inevitably, many other separate, small, and potentially dense components are brought together, there is cohesion and it becomes easier to manage. With so many moving parts it’s quite a different architectural model to the traditional, as over a period of just a few days, you could expand to tens of different services for ten different components. This is how easily your serverless footprint could grow.

For smaller footprints or those only used for prototyping, using native AWS services may be enough, however, once you’re at a handful of serverless components with other services like AWS S3 or AWS CloudTrail linked to them, your infrastructure will grow exponentially and observability tools become increasingly important

As an AWS Community Hero and part of the Serverless Leadership team, I see this issue raised time again with many people overlooking the importance and business value in instilling good practice and underestimating how quickly serverless architecture scales due to its very nature. 

One of the challenges of running and growing an unknown, bleeding edge platform is that there will always be gaps in there, which you don’t know about and even when you do find out, the problem might not ever have been solved before, meaning it comes down to needing a tool, like Dashbird, to identify the issue early on.

We knew we needed a monitoring and observability tool for our growing serverless infrastructure, but didn’t want to add strain to our own resources to build this.

The solution

Dashbird came out the winner in our trials because of its simplicity and agentless nature. It wasn’t intrusive and didn’t need any extra code changes to work. Instead, it ran side-by-side with our existing cloud infrastructure footprint, with cross-account access using a dedicated IAM role assigned that could easily be removed at any time. The straightforward dashboard and automated alerts meant that we were always aware of any failing statuses or issues, which allowed for much quicker fixes and fewer affected customers

serverless monitoring

As Cloud Conformity developed and the demand for cloud grew, the importance of ‘Shift Left’ became clearer. Preventing failures before deployment and essentially finding these errors earlier in the CI/CD pipeline is where the cloud is already moving, and I believe serverless will too. In addition, there is still a gap in full end-to-end monitoring within the space. 

Keep an eye on these two elements as I’m predicting more will happen and soon.

Made by Developers for Developers

Our history and present are deeply rooted in building large-scale cloud applications on state-of-the-art technology. Dashbird was born out of our own need for an enhanced serverless debugging and monitoring tool, and we take pride in being developers.

10,000+ developers trust Dashbird

Dashbird helped us reduce the time to discovery from 2-4 hours to a matter of seconds. This means we’re saving dozens of hours in developer time, which not only can be funneled into improving the product but also means hundreds of dollars are saved every month.

Great onboarding: it takes just a couple of minutes to connect an AWS account to an organization in Dashbird. The UI is clean and gives a good overview of what is happening with the Lambdas and API Gateways in the account.

Dashbird provides an easier interface to monitor and debug problems with our Lambdas. Relevant logs are simple to find and view. Dashbird’s support has been good, and they take product suggestions with grace.

Great UI. Easy to navigate through CloudWatch logs. Simple setup.

Dashbird helped us refine the size of our Lambdas, resulting in significantly reduced costs. We have Dashbird alert us in seconds via email when any of our functions behaves abnormally. Their app immediately makes the cause and severity of errors obvious.

Read our blog

AWS Step Functions Input and Output Manipulation Handbook

In this handbook, you’ll learn how to manipulate AWS Step Functions Input and Output and filter this data.

How to Save Hundreds of Hours on Lambda Debugging

Learn simple ways to save a ton of time when scanning logs to debug errors in your Lambda functions.

Why Are Some Engineers Missing The Point of Serverless?

Why are some engineers missing the point of serverless? Let’s have a look at the common critique points, benefits, drawbacks of serverless, and if it makes sense for your use case.

How Dashbird innovates serverless monitoring

What makes an effective serverless monitoring strategy? In this article, we’re discussing the three core ideas that Dashbird’s serverless monitoring tool was built on top and that should be the fundamentals of any effective serverless monitoring approach.

Debugging with Dashbird: Malformed Lambda Proxy Response

A problem that pops up quite frequently when people try to build serverless applications with AWS API Gateway and AWS Lambda is: Execution failed due to configuration error: Malformed Lambda proxy response.Learn what causes it and how to fix it.

Go to blog