The Circuit Breaker

Interactive Reference Architecture

Click on the components or numbered steps below to explore how this architecture works.

The Circuit Breaker pattern keeps track of the number of failed (or slow) API calls by using a cache to share the status across multiple Lambda functions. In this example, we're using a DynamoDB table so that we can avoid using a VPC. If you were in a VPC already, ElastiCache would be a good alternative.

Here's how it works. When the number of failures reaches a certain threshold, we "open" the circuit and send errors back to the calling client immediately without even trying to call the API. After a short period of time, we "half open" the circuit, sending just a few requests through to see if the API is finally responding correctly. All other requests receive an error. If the sample requests are successful, we "close" the circuit and start letting all traffic through. However, if some or all of those requests fail, the circuit stays "open", and the process repeats with some algorithm for increasing the timeout between "half open" retry attempts.

This is an incredibly powerful (and cost saving) pattern for any type of synchronous request to an API or downstream system. You are accumulating charges whenever a Lambda function is running and waiting for another task to complete. Allowing your systems to self-identify issues like this, provide incremental backoff, and then self-heal when the service comes back online, adds a tremendous amount of resiliency to your applications.

Deploy this Pattern

Below are the basic configurations for deploying this pattern using different frameworks and platforms. Additional configuration for your environment will be necessary. The source files and additional examples are available in the GitHub repo.

• • SAM

    EMBED DISABLED: https://app.stackery.io/editor/template?owner=serverless-architecture&repo=reference-architectures&file=circuit-breaker/sam/template.yaml&standalone=true&sync=stackeryEditorDesign

  • Stackery

    EMBED DISABLED: https://app.stackery.io/editor/design?owner=serverless-architecture&repo=reference-architectures&file=circuit-breaker/sam/template.yaml&standalone=true&sync=stackeryEditorTemplate

  • Serverless Framework

    yamlCopy code
    service: simple-web-service
    provider:
      name: aws
      runtime: nodejs12.x
      stage: ${opt:stage,'dev'}
      region: us-east-1
      stackName: ${self:service}-${self:provider.stage}
      stackTags:
        SERVICE: ${self:service}
      httpApi:
        payload: '2.0'
        cors: true
    
    functions:
      GetUser:
        handler: index.handler
        memorySize: 1024
        timeout: 6
        tracing: Active
        environment:
          TABLE_NAME: !Ref Users
          TABLE_ARN: !GetAtt Users.Arn
        iamRoleStatementsName: ${self:service}-${self:provider.stage}-getuser-role
        iamRoleStatements:
          - Effect: Allow
            Action:
              - dynamodb:getItem
              - dynamodb:putItem
              - dynamodb:updateItem
              - dynamodb:deleteItem
            Resource:
              - !Join [ '/', [ !GetAtt Users.Arn, '*' ] ]
              - !GetAtt Users.Arn
          - Effect: Allow
            Action:
              - xray:PutTraceSegments
              - xray:PutTelemetryRecords
            Resource: "*"
        events:
          - httpApi:
              path: /user/{id}
              method: get
    
    plugins:
      - serverless-iam-roles-per-function
    
    resources:
      Resources:
        Users:
          Type: AWS::DynamoDB::Table
          Properties:
            AttributeDefinitions:
              - AttributeName: id
                AttributeType: S
            BillingMode: PAY_PER_REQUEST
            KeySchema:
              - AttributeName: id
                KeyType: HASH
            StreamSpecification:
              StreamViewType: NEW_AND_OLD_IMAGES