Making Snowflakes: Avoiding Collisions in CloudFormation Stack Outputs

Generally speaking I’m pretty lucky because all our micro services operate in their own accounts (check out AWS Super Glue: The Lambda Function for why we do this) and I don’t have to worry too much about uniqueness of AWS resources. There are a few exceptions of course for resources that must be unique across the entire AWS platform, like S3 bucket names, and CloudFront CNAME’s. Since I’ve been so lucky I haven’t had to solve this problem of stack outputs and the requirement for them to be unique within an AWS account. Our micro services are fairly lightweight so I can get away with all my infrastructure existing within a single CloudFormation script. In my recent Web App Blue/Green deployment however I started running into stack outputs colliding.

Before we get too far into this, let’s take a step back and bring the CloudFormation rookies up to speed. If you just want to check out my solution, go ahead and skip down.

CloudFormation really is an incredibly powerful tool, and over the last few years AWS has really evolved the service into a very mature and feature rich offering. The essential goal of CloudFormation is to allow DevOps to design infrastructure as code, and to facilitate the automation of deployment and management of AWS infrastructure. For us, this has been a powerful and transformative change in our organization, since developers can now spin up, modify, and delete AWS infrastructure in a robust and secure way. No more raising a ticket to an IT or Operations team, and having to wait weeks or months for the rack and stack, installation, and licensing deployment cycle.

The documentation for CloudFormation is very well put together and gives a great view of what it can achieve. My only real complaint is because it’s an abstraction layer on top of the AWS API, the team has to ingest new features and resources after they are released, which means new configuration flags, or even new AWS resources can sometimes take a few months before being supported in CloudFormation. The ability to implement Lambda backed custom resources gives you the ability to develop and implement your own solutions, but the overhead to support this isn’t great, and I don’t really want to put in a bunch of work to develop support for a new AWS resource when it will eventually be supported in CloudFormation. If it’s just a configuration dip-switch it’s usually a month or two at most before it’s supported anyways.

Instead of giving you a crash course in CloudFormation, I’ll let you familiarize yourself with it first hand by checking out the documentation and the example CloudFormation stacks.

Back to the problem at hand.

One of the most powerful features of CloudFormation is the ability to output information like resource ARNs so you can reference those dynamically created resources in other stacks, or in my case referencing them in my deployment framework to help facilitate things like Blue/Green deployments.

Our stacks are very generic and use parameter inputs and conditions to behave slightly differently depending on the needs of the micro service or web app. This means I can develop, maintain, and implement a single stack that can be utilized by a number of teams. The downside to this generic approach, is that stack outputs get named the same and end up colliding if the stack is deployed more than once in the same account.

I don’t want to go down the path of modifying the stack for each implementation because now I’m starting to make stack snowflakes, and I try to give names that are useful so I don’t want to just append a random string or something to the end of the output name.

Here’s what a normal CloudFormation output looks like:

"CertificateArn": {
   "Description": "The ARN of the ACM certificate used for this deployment",
   "Export": {
      "Name": "CertificateArn"
   },
   "Value": {
      "Ref": "WebCert"
   }
}

You can see based on the template anatomy that we have an internal resource name “CertificateArn”, a description, the export name (that must be unique), and the value of the output. In my deployment work, I reference the internal resource name so my code doesn’t have to change based on the stack, but that export name still has to be unique across all instances of the stack. I want the name to be clear so I usually use the internal resource name as the export name. That doesn’t work when I implement the stack N number of times in the same account.

CloudFormation puts some restrictions on what a stack output name can be. From the documentation:

For outputs, the value of the Name property of an Export can’t use Ref or GetAtt functions that depend on a resource.

So that puts a bit of restriction on what I can do here. I can’t reference any of my stack input parameters, nor can I reference any internal resourcing, or conditions.

According to the documentation, I can however still utilize intrinsic functions in the export name.

You can use intrinsic functions to customize the Name value of an export.

The documentation actually gives a sample of exactly how I want to solve this problem. If I came to the same conclusion as AWS, I must be doing something right. So to solve the problem, we’re going to reference a pseudo parameter that I know has to be unique across all stacks…the stack name.

Here’s the same CloudFormation snippit in the new format

"CertificateArn": {
   "Description": "The ARN of the ACM certificate used for this 
   "Export": {
      "Name": {
         "Fn::Sub": "${AWS::StackName}:CertificateArn"
      },
      "Value": {
         "Ref": "WebCert"
      }
}

This gives us an output name in with the name of the stack, with a colon, and then the name CertificateArn.

platform-api-service:CertificateArn

The sample from the documentation uses an Fn::Join intrinsic function, but I actually find the readability of the Join function to be quite low, and you can do the same thing using a Substitution function.

Here’s what the above would look like using a Join and how much harder it is to comprehend what the end result would be

"CertificateArn": {
   "Description": "The ARN of the ACM certificate used for this deployment",
   "Export": {
      "Name": {
         "Fn::Join": [
            ":",
               [
                  {
                     "Ref": "AWS::StackName"
               },
               "CertificateArn"
               ]
         ]
      }
   },
   "Value": {
      "Ref": "WebCert"
   }
}

I think we can all agree the Fn::Sub function makes that a bit easier to read, and makes for a bit less code which is always a good thing.

So that’s it for this week. Good luck in your CloudFormation templates!

As always, if you’ve run into similar problems or are looking for a solution to a CloudFormation problem, shoot me an email anytime.

Until next time,

James.