Start free trial

Central data platform for your serverless environment.

Get full access to all premium features for 14 days. No code changes and no credit card required.

By signing up, you agree to our Privacy policy and Terms.

A Simple Introduction to AWS Step Functions

Share

Quick intro

Step Functions is a managed service by AWS that implements the Finite-state Machine (FSM) model.

You coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications.

You can read Wikipedia’s definition of a Finite-state Machine, but I think you’ll like the next section more. Keep on reading.

By example, we learn

Have you noticed that intersection traffic lights never turn green simultaneously for crossing directions? Not even once in a Million times?

It never bugs down! How come?

 

Thank FSM Gods next time you drive through it safely.

The FSM model ensures that, when a traffic light is about to go green for one direction, all the others turned red before.

It does that by managing states and transitions. Mark those two words. They’re pivotal.

Right, let’s clarify.

For one light to transition into the green state, all other lights must have transitioned to the red state before. Simple, right?

FSM is a robust model for that type of scenario: multiple states with transitional rules.

It’s kind of a design pattern. Not quite, but not too strechy.

Wanna see more?

Ever wondered how vending machines work? Yup, FSM is there too!

It guarantees you won’t get a snack until you feed it up with at least those 3 bucks. Ouch, expensive!

FSM will also manage the transition to delivering the snack after payment, and also secure your change if any is owed.

Yes, some times vending machines fail, but it’s not due to the FSM model

FSM is a mature and proven model that can be trusted. Implementing correctly isn’t hard. When done right, rest assured it is going to fulfill its promises.

FSM implementations

As a mature, battle-tested model, there are implementations in many programming languages, such as PythonJavascriptJava, etc.

Not recommending these libraries, just exemplifying. Do your research.

Or… even better… wait for it:

Wouldn’t there be a way to use FSM without having to implement anything programming-wise?

YES! Enters AWS Step Functions.

Step Functions: a managed FSM service

Following FSM concepts, AWS Step Functions also has states and transitions.

Tasks are also part of the package, but more on that later.

Straight to the point

We’ll start with some examples and go from there.

Consider a huge number of entries in a database. They all need to move to another storage location.

Too many entries, can’t do in a single shot. Will need to loop.

Step Functions example diagram

Step Functions implementation – example diagram – adapted from AWS Docs

This outlines how Step Functions would handle that data migration process for us.

The initial state is data seeded from the source database. It goes through a loop, then reading next entry, sending it to another location, until it finally succeeded.

It won’t magically guess what we need. An FSM is coded in Step Functions using the Amazon States Language. It’s just a JSON representation of your States, Transitions and Tasks.

Here’s the JSON snippet corresponding the diagram above.

Now Tasks. It’s what executes what you need once data transitions across states.

In our example, when a DB entry comes from the loop, it gotta go to the new storage location. A Task would involve, in simple terms:

  1. Getting the original data
  2. Formatting it for the new location standards
  3. Establishing a connection with the target database
  4. Inserting the data
  5. Waiting for confirmation
  6. Returning a 200 | Success response from the transfer request

In the AWS example, this Task is accomplished by a Lambda function.

Advantages of Step Functions

Auto-retry failures

In case the migration for a given entry fails for any reason, the Step Function can automatically retry it for us. Could even have it notifying ourselves in case of errors.

It will make sure an entry is not duplicated in the target storage by tracking its state and whether a retry is needed.

Once successfully migrated, an entry cannot possibly go back to “pending migration” state. That’s guaranteed by the FSM model.

FSM Benefits

Everything beautiful about the FSM model comes bundled by default:

  • Maturity and trustworthiness: resilient and fault-tolerant
  • Flexibility
  • Quickly move from an abstract, conceptual process to code and execution
  • Little processing overhead

Adapted from: elprocus.com

Fully-managed

As we discussed previously, we can avoid the hassle of implementing an FSM library in our own environment.

AWS will manage everything. Make sure servers are provisioned to run our state machines. Scale the infrastructure when needed. Up and down. Horizontally. In the fourth dimension. Kidding.

No, but really, scalability is difficult to get done right, this is a big plus.

Integrations

Step Functions will seamlessly integrate with various other AWS services.

It’s possible to connect your Step Functions workflow with other AWS services or even other workflows. That’s right, you can compose workflows to create higher-level processes. Here are some examples of what you can do:

  • Invoke a Lambda function
  • Run Amazon Fargate task or Amazon ECS
  • Submit an AWS batch job
  • Publish a message to an SNS Topic
  • Interact with SageMaker machine learning model training, inference and classification
  • Send a message to an SQS queue
  • Put or get items from a DynamoDB table

Monitoring & Debugging

You must be asking: ok, this all too good, but how do I monitor my workflows, make sure they’re running as expected or debug issues?

Step Functions publishes events and metrics to CloudTrail and CloudWatch, and is also monitored by Dashbird’s Serverless Insights platform which contributes to better architectural practices. With Dashbird you can set alarms and monitor the health of workflows throught various indicators.

Made by Engineers for Engineers

Our history and present are deeply rooted in building large-scale cloud applications on state-of-the-art technology. Dashbird was born out of our own need and we take pride in being developers.

Trusted by hundreds

https://mk0dashbirdiotlcuqbg.kinstacdn.com/wp-content/uploads/2020/10/gus-gordon__1_.png

Gus Gordon

Dashbird helped us refine the size of our Lambdas, resulting in significantly reduced costs. We have Dashbird alert us in seconds via email when any of our functions behaves abnormally. Their app immediately makes the cause and severity of errors obvious.

Read our blog

The Ultimate Guide to AWS Step Functions

The use of serverless computing has become a must nowadays, and some of you may already know a thing or two about Amazon Web Services like Lambda Functions, Step Functions, and other services AWS provides. However, if this is the first time you hear about them – fantastic!

In this article, we’ll discuss AWS Step Functions, what they are used for, how to use them, and the advantages or disadvantages that they bring.

Best Practices For Logging In AWS Lambda

We cover the best practices when logging within AWS Lambda. Explaining how and why these ways will simplify your AWS Lambda logging.

 

Create your first website with serverless in 15 minutes

Alright, we talked the “serverless is awesome” talk but now it’s about time to walk the walk. In the light of us launching our brand new website, we thought it would be fun to show you how to create a serverless website from scratch. In the next 15 minutes, we’ll take you through the entire process of creating your first website running on serverless and you better believe it’s going to be awesome.

The Complete AWS Lambda Handbook for Beginners (Part 3)

Welcome to the final installment of our Complete AWS Lambda Handbook series! In part three of our series, we’ll cover Lambda deployment limitations, deployment package testing, cold starts, as well as observability, debugging, and monitoring AWS Lambda.

 

The Ultimate Guide to AWS DynamoDB

AWS DynamoDB changed the database game in Serverless and continues to do so, as its design repeatedly proves its huge value. In this post, you’ll find out everything you need to know to get started with DynamoDB.