Errors and Fixes using AWS CloudFormation to Deploy Shell Scripts via AWS Systems Manager Associations

While developing a CloudFormation template recently, I received a variety of different error messages from the AWS CloudFormation service. These errors predominantly were centered around the AWS Systems Manager service, which enables you to perform automation tasks. One such automation task that I wanted to perform was to deploy a shell script to an Amazon EC2 instance running Ubuntu Linux.

AWS Systems Manager provides a native / out-of-box AWS Systems Manager “document” called AWS-RunRemoteScript. This “document” can deploy a shell script to an Amazon EC2 instance from two different data sources: 1) a public or private GitHub repository, or 2) an Amazon S3 bucket.

Below, I will share a few different error messages I observed, and resolved, while developing a CloudFormation template that utilizes this AWS Systems Manager “document.”

Invalid Parameter Definitions for AWS Systems Manager Document

When invoking an AWS Systems Manager “document,” you can specify input parameters that the document itself declares. To view a list of parameters supported by a given “document,” navigate to the AWS Systems Manager feature in the AWS Management Console, then go to Shared Resources –> Documents, and search for the Document that you want to use. Click into its detail screen, and there should be a Parameters tab that describes each of the parameters that the document is configured to accept.

At first, when calling the AWS-RunRemoteScript document, I tried using the following parameter definitions in my CloudFormation template.

  SSMScript:
    Type: AWS::SSM::Association
    Properties:
      Name: AWS-RunRemoteScript
      Parameters:
        sourceType: "S3"
        sourceInfo: [!Sub '{"path": "https://s3.amazonaws.com/${S3BucketName}/artofshell/scripts/prereq.sh"}']
        commandLine: "prereq.sh"
...

This resulted in the following error message.

Value of {Parameters} must be a map where each value is a list of {String}

Although this error message is rather obtuse, you’ll notice that it specifically mentions accepting a list of string elements. In my first iteration, I had specified a singleton string value (think in JSON), instead of an array (aka. list, in this case).

To correct this error message, even though it doesn’t really seem sensible to do so, go ahead and wrap the parameter values in square brackets, to turn them into a JSON array. Check out the example below for a working sample. It’s a rather subtle change, but the Systems Manager service is expecting an array (list) of string values, even though it doesn’t make sense to specify an array for certain parameters. In particular, in my case where I’m using AWS-RunRemoteScript document, the sourceType parameter can only be either GitHub or S3, not both. Nonetheless, the CloudFormation template wants this to be a JSON array value.

  SSMScript:
    Type: AWS::SSM::Association
    Properties:
      Name: AWS-RunRemoteScript
      Parameters:
        sourceType: ["S3"]
        sourceInfo: [!Sub '{"path": "https://s3.amazonaws.com/${S3BucketName}/artofshell/scripts/prereq.sh"}']
        commandLine: ["prereq.sh"]
...

Targeting EC2 Instances by InstanceId

When utilizing AWS Systems Manager, you can “target” EC2 instances based on their specific InstanceId attribute, or by targeting many EC2 instances by segments, using an AWS tagging strategy. During my first iteration, I tried using the InstanceId property in my CloudFormation YAML, to target a specific EC2 instance.

For example, I had a CloudFormation resource that looked like this:

  SSMScript:
    Type: AWS::SSM::Association
    Properties:
      AssociationName: artofshell-trevor
      InstanceId: !Ref EC2Instance
      Name: AWS-RunRemoteScript
      Parameters:
...
...

Upon deploying this template, I received the following error:

Document schema version, 2.2, is not supported by association that is created with instance id (Service: AWSSimpleSystemsManagement; Status Code: 400; Error Code: InvalidDocument; Request ID: a738f00d-cbb8-4477-99cc-290ad39b75b1)

To fix this error, I had to use the Targets property instead of the InstanceId property on the AWS::SSM::Association resource in my CloudFormation template.

  SSMScript:
    Type: AWS::SSM::Association
    Properties:
      AssociationName: artofshell-trevor
      Targets:
        - Key: tag:Purpose
          Values: 
            - ArtofShell
      Name: AWS-RunRemoteScript
...

Invalid Cron ScheduleExpression

In AWS Systems Manager’s State Manager feature, you can deploy an automation task on a schedule. You can expression this schedule using either a rate(…) or cron(…) expression. However, you must be cognizant of the precise syntax and constraints to define one of these interval expressions. Use the official AWS documentation to help guide your understanding of these intervals.

During my first attempt at declaring a Systems Manager State Manager “Association” in CloudFormation YAML syntax, I received the error message below.

  SSMScript:
    Type: AWS::SSM::Association
    Properties:
      ScheduleExpression: cron(0/15 * * * ? *)
...

Schedule expression cron(0/15 * * * ? *) is currently not accepted. Supported expressions are every half, 1, 2, 4, 8 or 12 hour(s), every specified day and time of the week. Supported examples are: cron(0 0/30 * 1/1 * ? *), cron(0 0 0/4 1/1 * ? *), cron (0 0 10 ? * SUN *), cron (0 0 10 ? * * *) (Service: AWSSimpleSystemsManagement; Status Code: 400; Error Code: InvalidSchedule; Request ID: 44920694-af81-43d2-81c4-1571c7d62aca)

To resolve this error message, I had to update my ScheduleExpression property, on my AWS::SSM::Association resource, to cron(0/30 * * * ? *). It seems that Systems Manager has a minimum of 30-minute intervals, when using cron expressions. Although I didn’t test it out, you might be able to use a smaller time interval with a rate(…) expression instead, such as every 5 or 10 minutes.

You can read more about the AWS::SSM::Association resource’s CloudFormation definition in the official AWS documentation.

  SSMScript:
    Type: AWS::SSM::Association
    Properties:
      ScheduleExpression: cron(0/30 * * * ? *)
...

Conclusion

As you can see, using trial and error, I was able to identify the source of my CloudFormation errors around State Manager “Associations,” and eventually resolve them. With enough determination and hacking prowess, it’s relatively straightforward to figure out what the CloudFormation service is expecting for input. However, the official AWS documentation could still be improved to cover more scenarios by example.

In summary, here’s what the final snippet looks like for my AWS Systems Manager association, leveraging the native AWS-RunRemoteScript document. Notice that my example uses an Amazon S3 bucket as the data source, whereas you may want to change the data source properties to use GitHub instead.

I hope this post helps you utilize the AWS Systems Manager service and spend less time troubleshooting error messages in your CloudFormation templates that leverage this service.

  SSMScript:
    Type: AWS::SSM::Association
    Properties:
      AssociationName: artofshell-trevor
      #InstanceId: !Ref EC2Instance
      Name: AWS-RunRemoteScript
      Parameters:
        sourceType: ["S3"]
        sourceInfo: [!Sub '{"path": "https://s3.amazonaws.com/${S3BucketName}/artofshell/scripts/prereq.sh"}']
        commandLine: ["prereq.sh"]
      ScheduleExpression: cron(0/30 * * * ? *)
      Targets:
        - Key: tag:Purpose
          Values: 
            - artofshell