dougmoscrop/serverless-plugin-split-stacks

Only the first function in my serverless.yml is placed in a nested stack with the VPC defined as a parameter

william-matz opened this issue · 1 comments

Serverless version

% sls version
Framework Core: 1.83.3 (local)
Plugin: 3.8.4
SDK: 2.3.2
Components: 2.34.9

serverless-plugin-split-stacks version

"serverless-plugin-split-stacks": "^1.11.2",

stacks-map.js
All the lambda function resources get resolved into stacks based on their domain

module.exports = (resource, logicalId) => {
  if (logicalId.startsWith("ServerlessDeploymentBucket")) {
    return;
  }
  if (logicalId.startsWith("Lambda")) {
    return;
  }
  if (logicalId.startsWith("ApiGatewayMethod")) {
    const firstKey = logicalId
      .replace("ApiGatewayMethod", "")
      .split(/(?=[A-Z])/)[0];
    return {
      destination: firstKey,
    };
  }
  if (logicalId.startsWith("ApiGatewayResource")) {
    const firstKey = logicalId
      .replace("ApiGatewayResource", "")
      .split(/(?=[A-Z])/)[0];
    return {
      destination: firstKey,
    };
  }
  if (
    logicalId.startsWith("ApiGatewayDeployment") ||
    logicalId.startsWith("Migration") ||
    resource.Type === "AWS::ApiGateway::GatewayResponse" ||
    resource.Type === "AWS::IAM::Role" ||
    logicalId.startsWith("Serverless") ||
    logicalId.startsWith("Website") ||
    logicalId.startsWith("Migration") ||
    logicalId.startsWith("Response") ||
    logicalId.startsWith("ApiGateway")
  ) {
    return;
  }
  if (logicalId.startsWith("Auth")) {
    return {
      destination: "Auth",
    };
  }
  if (logicalId.startsWith("WarmUp")) {
    return {
      destination: "WarmUp",
    };
  }
  const firstKey = logicalId.split("Dash")[0];
  return { destination: firstKey };
};

Here's a snippet from our serverless.yml file, defining configs for the split-stacks config and our VPC config, which reference VPC resources defined in external files

provider:
  vpc:
    securityGroupIds:
      - !GetAtt ServerlessSecurityGroup.GroupId
    subnetIds:
      - Ref: ServerlessSubnetA
      - Ref: ServerlessSubnetB
      - Ref: ServerlessSubnetC
 ...
 custom:
  splitStacks:
    perFunction: true
    perType: false
    perGroupFunction: false

When I run sls package with only one function in my serverless file, it correctly adds the referenced parameters in the parameters of the stack

{
  "Parameters": {
    "ServerlessSubnetCParameter": {
      "Type": "String"
    },
    "ServerlessSubnetBParameter": {
      "Type": "String"
    },
    "ServerlessSubnetAParameter": {
      "Type": "String"
    },
    "ServerlessSecurityGroupGroupIdParameter": {
      "Type": "String"
    },
  },
  "Resources": {
    "Function1": {
      "Properties": {
        "VpcConfig": {
          "SecurityGroupIds": [
            {
              "Ref": "ServerlessSecurityGroupGroupIdParameter"
            }
          ],
          "SubnetIds": [
            {
              "Ref": "ServerlessSubnetAParameter"
            },
            {
              "Ref": "ServerlessSubnetBParameter"
            },
            {
              "Ref": "ServerlessSubnetCParameter"
            }
          ]
        }
      },
    }
  },
}

But as soon as I have two or more resources, the subsequent stacks do not have the parameters referenced, and the stack with the first function still does reference those parameters

{
  "Parameters": {
    ... A few parameters, but not ServerlessSubnetAParameter, etc
  },
  "Resources": {
    "Function2": {
      "Properties": {
        "VpcConfig": {
          "SecurityGroupIds": [
            {
              "Ref": "ServerlessSecurityGroupGroupIdParameter"
            }
          ],
          "SubnetIds": [
            {
              "Ref": "ServerlessSubnetAParameter"
            },
            {
              "Ref": "ServerlessSubnetBParameter"
            },
            {
              "Ref": "ServerlessSubnetCParameter"
            }
          ]
        }
      },
    }
  },
}

I have 128 lambda functions I'm working on deploying
Here's what I've tried to deploy:

  1. The current setup
    I get the following error message:
Template format error: Unresolved resource dependencies [ServerlessSecurityGroupGroupIdParameter, ServerlessSubnetAParameter, ServerlessSubnetBParameter, ServerlessSubnetCParameter] in the Resources block of the template
  1. Sending all API gateway and Lambda Function resources to the first stack
  if (resource.Type === "AWS::Lambda::Function") {
    return { destination: "TestStack" };
  }

Doesn't work, circular dependency error

Error: The CloudFormation template is invalid: Circular dependency between resources: [TestStackNestedStack, TestFunctionNestedStack]
  1. Hard-code the deployed VPC resource ids - everything works fine, but it defeats the purpose of CF :(

Here's what I'm wondering:

  1. Is this a bug or expected behavior?
  2. Is there any way I can forcefully inject those parameters into the generated stacks?
  3. Is there any other setup (splitting technique) I can use here to solve this issue?

Update: I believe the root cause is described here: serverless/serverless#7206
Specifically, this comment: serverless/serverless#7206 (comment)

Switching my serverless version down to 1.59 solved this
Leaving this issue open because it seems like something this plugin should account for, though I don't understand fully understand what's going on