Skip to content

When validate=true, Map state machines in DISTRIBUTED mode incorrectly throw a yaml validation error #540

Open
@robtoth

Description

@robtoth

This is a (Bug Report / Feature Proposal)

Description

Attempting to deploy a state machine with Map in Distributed mode:
https://docs.aws.amazon.com/step-functions/latest/dg/concepts-asl-use-map-state-distributed.html

serverless.yml

...(events, etc.)
      definition:
        StartAt: MyMapState
        States:
          MyMapState:
            Type: Map
            ItemProcessor:
              StartAt: My map task
              States:
                My map task:
                  Type: Task
                  Resource:
                    Fn::GetAtt: [mylambda, Arn]
                  End: true
              ProcessorConfig:
                Mode: 'DISTRIBUTED'
                ExecutionType: 'STANDARD'
            End: true
  validate: true

This fails:

**$ sls deploy**
Running "serverless" from node_modules

Deploying project to stage dev (us-east-1)

✖ Stack project failed to deploy (18s)
Environment: linux, node 12.22.9, framework 3.24.1 (local) 3.24.1v (global), plugin 6.2.2, SDK 4.3.2
Credentials: Local, "semler-iac" profile
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
✕ State machine "MyMapState" definition is invalid:
SCHEMA_VALIDATION_FAILED: /States/MyMapState is invalid. must match exactly one schema in oneOf

However, when I turn the validator off and let it try to update on AWS cloudformation, it succeeds:

...(events, etc.)
      definition:
        StartAt: MyMapState
        States:
          MyMapState:
            Type: Map
            ItemProcessor:
              StartAt: My map task
              States:
                My map task:
                  Type: Task
                  Resource:
                    Fn::GetAtt: [mylambda, Arn]
                  End: true
              ProcessorConfig:
                Mode: 'DISTRIBUTED'
                ExecutionType: 'STANDARD'
            End: true
  validate: false
$ sls deploy
Running "serverless" from node_modules

Deploying semler-measurement-quality to stage dev (us-east-1)

✔ Service deployed to stack semler-measurement-quality-dev (121s)

Notes

It's important to realize that when in distributed map state, "Iterator" must be replaced with "ItemProcessor"
https://docs.aws.amazon.com/step-functions/latest/dg/concepts-asl-use-map-state-distributed.html#map-state-distributed-additional-fields

I'm assuming the validator here in this plugin

if (this.serverless.service.stepFunctions.validate) {

just pulls
calls the main serverless validator function here
https://github.com/serverless/serverless/blob/906ea319dd1486f79d6e088a999fd5634526c4bc/lib/classes/config-schema-handler/index.js#L77
which somehow compares the known schema to something (pulled from AWS?).

That's as far as I was able to dig into the code. I couldn't find where the schema for the "ItemProcessor" yaml section was being defined, perhaps on AWS cloud or perhaps I just couldn't find it in the code.

But regardless, it seems that the validator expects only Iterator instead of ItemProcessor and therefore will not allow the user to use the map in distributed mode.

Additional Data

$ sls --version
Running "serverless" from node_modules
Framework Core: 3.24.1 (local) 3.24.1 (global)
Plugin: 6.2.2
SDK: 4.3.2
cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"
cat package.json 
{
  "devDependencies": {
    "serverless": "^3.22.0",
    "serverless-iam-roles-per-function": "^3.2.0",
    "serverless-python-requirements": "^6.0.0",
    "serverless-step-functions": "^3.11.1"
  }
}

(Error messages provided above)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions