A Simple Introduction to AWS Step Functions

AWS Step Functions is a managed service by AWS that implements the Finite-state Machine (FSM) model.

You coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications.

You can read Wikipedia’s definition of a Finite-state Machine, but I think you’ll like the next section more. Keep on reading.

By example, we learn!

Have you noticed that intersection traffic lights never turn green simultaneously for crossing directions? Not even once in a million times?

It never bugs down! How come?

Thank FSM Gods next time you drive through it safely.

The FSM model ensures that, when a traffic light is about to go green for one direction, all the others turned red before.vIt does that by managing states and transitions. Mark those two words. They’re pivotal.

Right, let’s clarify.

For one light to transition into the green state, all other lights must have transitioned to the red state before. Simple, right?

FSM is a robust model for that type of scenario: multiple states with transitional rules. It’s kind of a design pattern. Not quite, but not too strechy.

Wanna see more?

Ever wondered how vending machines work? Yup, FSM is there too! It guarantees you won’t get a snack until you feed it up with at least those 3 bucks. Ouch, expensive! FSM will also manage the transition to delivering the snack after payment, and also secure your change if any is owed.

Yes, some times vending machines fail, but it’s not due to the FSM model

FSM is a mature and proven model that can be trusted. Implementing correctly isn’t hard. When done right, rest assured it is going to fulfill its promises.

FSM implementations

As a mature, battle-tested model, there are implementations in many programming languages, such as PythonJavascriptJava, etc.

Please do your own research when deciding on libraries as these are just examples.

Or… even better… wait for it:

Wouldn’t there be a way to use FSM without having to implement anything programming-wise?

YES! Enter AWS Step Functions.

Step Functions: a managed FSM service

aws step functions

Following FSM concepts, AWS Step Functions also has states and transitions. Tasks are also part of the package, but more on that later.

Straight to the point

We’ll start with some examples and go from there. Consider a huge number of entries in a database. They all need to move to another storage location. There are too many entries, we can’t do in a single shot. Will need to loop.

Step Functions example diagram

Step Functions implementation – example diagram – adapted from AWS Docs

This outlines how Step Functions would handle that data migration process for us.

The initial state is data seeded from the source database. It goes through a loop, then reading the next entry, sending it to another location, until it finally succeeded.

It won’t magically guess what we need. An FSM is coded in Step Functions using the Amazon States Language. It’s just a JSON representation of your States, Transitions, and Tasks.

Here’s the JSON snippet corresponding to the diagram above.

Now Tasks. It’s what executes what you need once data transitions across states.

In our example, when a DB entry comes from the loop, it gotta go to the new storage location. A Task would involve, in simple terms:

  1. Getting the original data
  2. Formatting it for the new location standards
  3. Establishing a connection with the target database
  4. Inserting the data
  5. Waiting for confirmation
  6. Returning a 200 | Success response from the transfer request

In the AWS example, this Task is accomplished by a Lambda function.

Advantages of Step Functions

Auto-retry failures

In case the migration for a given entry fails for any reason, the Step Function can automatically retry it for us. Could even have it notifying ourselves in case of errors.

It will make sure an entry is not duplicated in the target storage by tracking its state and whether a retry is needed.

Once successfully migrated, an entry cannot possibly go back to “pending migration” state. That’s guaranteed by the FSM model.

FSM Benefits

Everything beautiful about the FSM model comes bundled by default:

  • Maturity and trustworthiness: resilient and fault-tolerant
  • Flexibility
  • Quickly move from an abstract, conceptual process to code and execution
  • Little processing overhead

Adapted from: elprocus.com

Fully-managed

As we discussed previously, we can avoid the hassle of implementing an FSM library in our own environment.

AWS will manage everything. Make sure servers are provisioned to run our state machines. Scale the infrastructure when needed. Up and down. Horizontally. In the fourth dimension. Kidding.

No, but really, scalability is difficult to get done right, this is a big plus.

Integrations

Step Functions will seamlessly integrate with various other AWS services.

It’s possible to connect your Step Functions workflow with other AWS services or even other workflows. That’s right, you can compose workflows to create higher-level processes. Here are some examples of what you can do:

  • Invoke a Lambda function
  • Run Amazon Fargate task or Amazon ECS
  • Submit an AWS batch job
  • Publish a message to an SNS Topic
  • Interact with SageMaker machine learning model training, inference, and classification
  • Send a message to an SQS queue
  • Put or get items from a DynamoDB table

Monitoring & Debugging

You must be asking: ok, this all too good, but how do I monitor my workflows, make sure they’re running as expected, or debug issues?

monitor step functions

Step Functions publishes events and metrics to CloudTrail and CloudWatch, and is also monitored by Dashbird’s Step Functions monitoring platform which contributes to better architectural practices. With Dashbird you can set alarms and monitor the health of workflows throughout various indicators.

Read our blog

ANNOUNCEMENT: new pricing and the end of free tier

Today we are announcing a new, updated pricing model and the end of free tier for Dashbird.

4 Tips for AWS Lambda Performance Optimization

In this article, we’re covering 4 tips for AWS Lambda optimization for production. Covering error handling, memory provisioning, monitoring, performance, and more.

AWS Lambda Free Tier: Where Are The Limits?

In this article we’ll go through the ins and outs of AWS Lambda pricing model, how it works, what additional charges you might be looking at and what’s in the fine print.

Made by developers for developers

Dashbird was born out of our own need for an enhanced serverless debugging and monitoring tool, and we take pride in being developers.

What our customers say

Dashbird gives us a simple and easy to use tool to have peace of mind and know that all of our Serverless functions are running correctly. We are instantly aware now if there’s a problem. We love the fact that we have enough information in the Slack notification itself to take appropriate action immediately and know exactly where the issue occurred.

Thanks to Dashbird the time to discover the occurrence of an issue reduced from 2-4 hours to a matter of seconds or minutes. It also means that hundreds of dollars are saved every month.

Great onboarding: it takes just a couple of minutes to connect an AWS account to an organization in Dashbird. The UI is clean and gives a good overview of what is happening with the Lambdas and API Gateways in the account.

I mean, it is just extremely time-saving. It’s so efficient! I don’t think it’s an exaggeration or dramatic to say that Dashbird has been a lifesaver for us.

Dashbird provides an easier interface to monitor and debug problems with our Lambdas. Relevant logs are simple to find and view. Dashbird’s support has been good, and they take product suggestions with grace.

Great UI. Easy to navigate through CloudWatch logs. Simple setup.

Dashbird helped us refine the size of our Lambdas, resulting in significantly reduced costs. We have Dashbird alert us in seconds via email when any of our functions behaves abnormally. Their app immediately makes the cause and severity of errors obvious.