aws-cloudformation/cfn-language-discussion

Fn::JsonString

Closed this issue ยท 14 comments

Sviat commented

1. Title

Fn::JsonString

2. Scope of request

JSON CloudFormation Templates.

Whenever a resource has property that requires content as JSON String, we all either write this Fn::Join's with "\n" and other workarounds, or convert JSON to String by external tool.

This lowers readability of code (Infrastructure is a Code!) to zero, and highers maintenance costs.
Example:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudwatch-dashboard.html DashboardBody

3. Expected behavior

Instead of trying to construct string out of JSON key-values and Fn::Ref's / Fn::GetAtt's, should be possible to assign property JSON object and hope CloudFormation will handle the rest.

Example:

"Resources": { "Dashboard": { "Type" : "AWS::CloudWatch::Dashboard", "Properties" : { "DashboardName" : "Lorem Ipsum 42", "DashboardBody" : { "Fn::JsonString": { "Key1": "Value", "Key2": { "Fn::Ref": "SomeId" }, "Key3": { ... } } }

4. Suggest specific test cases

See 3.

5. Helpful Links to speed up research and evaluation

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-join.html

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudwatch-dashboard.html#aws-resource-cloudwatch-dashboard-properties

People do not deserve such workarounds https://stackoverflow.com/questions/39041209/how-to-specify-json-formatted-string-in-cloudformation

6. Category

  1. Other (IoT, Migration, Budgets...) - Intrinsic Functions
jk2l commented

+1 on this.

Use case for me, rather than try to write and escape a string in json format. i will prefer I can let a function automate it

  # FROM 
  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub "${SecretNamespace}"
      Description: "This is a shared secret."
      SecretString:
        Fn::JsonString:
          username: "test-user"
      KmsKeyId: !Ref SecretKmsKey

 # TO
  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub "${SecretNamespace}"
      Description: "This is a shared secret."
      SecretString:  "{\"username\":\"test-user\"}"
      KmsKeyId: !Ref SecretKmsKey

I lost count how many time i have some copy and paste issue or some syntax issue that break the JSON validation. having a function to automate it will be a great feature

Not to take away from this very valid request, but you can accomplish this a little bit today, using YAML's multiline syntax. For example:

  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub "${SecretNamespace}"
      Description: "This is a shared secret."
      SecretString: !Sub >-
        {"username": "${UserName}"}
      KmsKeyId: !Ref SecretKmsKey
jk2l commented

that is actually a creative use of it. hm.. interesting. thanks will give that a go

@benkehoe Can you also explain what it actually is doing here?

jk2l commented

@benkehoe Can you also explain what it actually is doing here?

https://yaml-multiline.info/

the >- is passing down the whole block as raw string. this is different to a double quote style that the block itself is string. so no need to do escape on the quote within the block.

this is a combination of Block Style Indicator > and Block Chomping Indicator - as the link I provided. > will turn your multiline string into compressed format as it replace newline characters to space. and - declare no newline at the end of the string block.

Okay, so everything inside the >- block is treated as a raw string. I have some questions then:

  • What is the significance of the ${UserName} here? (I know it's related to !Sub method of CF, but not sure how this works here.)
  • How would you write your JSON using this method? Your JSON has a "test-user", but I don't understand how it is present in this method.

Thanks for the answer though!

jk2l commented

Okay, so everything inside the >- block is treated as a raw string. I have some questions then:

* What is the significance of the `${UserName}` here? (I know it's related to `!Sub` method of CF, but not sure how this works here.)

* How would you write your JSON using this method? Your JSON has a "test-user", but I don't understand how it is present in this method.

Thanks for the answer though!

  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub "${SecretNamespace}"
      Description: "This is a shared secret."
      SecretString: !Sub >-
        {"username": "${UserName}"}
      KmsKeyId: !Ref SecretKmsKey

Let say I have a 3rd party service i need to integrate. i can pass down the username of it into the cloudformation via a parameter.

so in dev i have parameter UserName = "DevUser"
in prod i have parameter UserName = "User"

Sub function will subsitute your ${UserName} with whatever the value it is from the parameter

Note that there are two forms of !Sub. In this form, where only a string is provided, the names of the variables (i.e., UserName in this case) have to be valid names in the template, i.e., resource names, parameter names, pseudoparameters (e.g., AWS::AccountId) or resource attributes if you use ResourceName.AttributeName. The other form, which is useful if the values you want to insert need to be manipulated in some way (e.g., using a !Join or conditionally with !If, takes a list, the first element of which is the string, and the second element is a map from variable names to variable values (note: you can still use names from the template directly, without providing them in the variable map).

Docs page: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html

Totally support this. It is one of the crappiest things about CFN.

A part of this I am not seeing mentioned is validating the json string. With more complex json documents, and embedding them in a yaml cfn template, the validation that could be offered by a json encoding function would be amazing. Or even being able to write the object in yaml, so cloudformation has to load and validate the yaml, while still serializing it to json.

  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: !Sub "${SecretNamespace}"
      Description: "This is a shared secret."
      SecretString: !Sub >-
          {"username": "${UserName}"},         # <--- oops, a trailing comma!
      KmsKeyId: !Ref SecretKmsKey

@Sviat Thank you very much for your feedback! Since this repository is focused on resource coverage, I'm transferring this issue over to a new GitHub repository dedicated to CloudFormation template language issues.

Hey all, we have created an RFC to address this problem. We invite you to have your say there as well:
RFC 0014: New Intrinsic Function to Convert Template Block to JSON String

Hi all, really appreciate all the feedback and feature proposal. I'm excited that share that now you can now convert JSON objects into JSON string with Fn::ToJsonString (more details can be found here) now. Please try it out and feel free to leave your comments here!

Since Fn:ToJsonString is now available, we can close this issue