Major: rework environment configuration of CDK

[simonrw]

Motivation

We have had numerous reports of users trying to use cdklocal and receiving error messages like:

S3Stack: SSM parameter /cdk-bootstrap/hnb659fds/version not found. Has the environment been bootstrapped? Please run 'cdk bootstrap

or

Environment aws://000000000000/eu-west-2 failed bootstrapping: Error: Need to perform AWS calls for account 000000000000, but no credentials have been configured0w

My hypothesis is when cdk switched to using the JavaScript AWS SDK v3 our patching/envar overriding in cdklocal reads valid AWS configuration from the environment, and either does not execute against LocalStack correctly, or tries to deploy to AWS but does not have enough context to connect correctly.

In #107 we updated cdklocal for cdk >= 2.177.0 by configuring just AWS_ENDPOINT_URL and AWS_ENDPOINT_URL_S3. However we still read e.g. the AWS_PROFILE envar, or AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY envars. This confusion of environment variables and configuration should be removed - we should sanitise the environment in cdklocal before executing commands.

Future work

We are starting to evaluate deprecating our "*local" tools and using the first party tooling where possible. In this issue comment I outline a working configuration for using the upstream cdk with LocalStack. In summary, creating an AWS profile (in ~/.aws/config) with the following contents:

[profile localstack]
region = us-east-1
endpoint_url = https://localhost.localstack.cloud:4566
aws_access_key_id = test
aws_secret_access_key = test
cli_binary_format=raw-in-base64-out
services = localstack-services

[services localstack-services]
s3 =
  endpoint_url = http://s3.localhost.localstack.cloud:4566

Changes

This PR includes a few changes (#srynotsry)

• Testing improvements
  • Creates a utility library for helpers in cdklocal primarily so we can unit test
  • Moves configureEnvironment to the utilities library for testing
  • Adds unit tests for configureEnvironment
  • Adds CI step for running unit tests
  • Ensures only one commit's CI jobs run at once per branch/PR
• Environment sanitisation
  • Raise an exception if the user specifies AWS_ENDPOINT_URL without AWS_ENDPOINT_URL_S3 as we cannot automatically determine a configuration that will work. We need to ensure that .s3. is in the domain for S3 requests. If the user customises the endpoint then it may not support subdomains (so we could not just inject .s3. in their URL, and we cannot use the same URL as our S3 detection won't work.
  • (by default) removes all AWS_* envars from the environment before executing the CDK, except:
    • AWS_ENDPOINT_URL
    • AWS_ENDPOINT_URL_S3
    • AWS_REGION
    • AWS_DEFAULT_REGION
  • Provides an override (AWS_ENVAR_ALLOWLIST) to allow specific envars past this sanitsation, for example if the user wants to deploy to a different region
  • Set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION and AWS_DEFAULT_REGION if customised
• Versioning
  • Bump the version of the tool to 3.0.0 as this change requires changes from the user to maintain the current (i.e. potentially broken) behaviour

Testing

• Set up your shell to target AWS (i.e. so that aws sts get-caller-identity returns your AWS credentials) and run cdklocal bootstrap && cdklocal deploy --require-approval never to deploy your CDK stack. The stack should be deployed successfully to LocalStack.

[pinzon]

LGTM. I hope this finally fixes that weird issue. 👍

[dominikschubert]

LGTM, thanks for the thorough documentation around sanitizing the environment variables. 🚀

Had a few minor nits but nothing blocking us from getting this out

[dominikschubert]

suggestion: could use the newest version (v4) if there's no specific reason to stay on a lower one

[dominikschubert]

suggestion: same as above, newest version for that action is v4

[dominikschubert]

Suggested change

1. We do this because other environment variables may lead to a conflicting set of configuration options, where the wrong region is used to target LocalStack, or `cdklocal` tries to deploy into upstream AWS by mistake. If individual configuration variables are needed for the deploy process (e.g. `AwS_REGION`) these configuration variables can be propagated to the `cdk` command by configuring `AWS_ENVAR_ALLOWLIST`, for example: `AWS_ENVAR_ALLOWLIST=AWS_REGION,AWS_DEFAULT_REGION AWS_REGION=eu-central-1 cdklocal ...`.

1. We do this because other environment variables may lead to a conflicting set of configuration options, where the wrong region is used to target LocalStack, or `cdklocal` tries to deploy into upstream AWS by mistake. If individual configuration variables are needed for the deploy process (e.g. `AWS_REGION`) these configuration variables can be propagated to the `cdk` command by configuring `AWS_ENVAR_ALLOWLIST`, for example: `AWS_ENVAR_ALLOWLIST=AWS_REGION,AWS_DEFAULT_REGION AWS_REGION=eu-central-1 cdklocal ...`.

also: Isn't the AWS_DEFAULT_REGION in the allowlist example a bit confusing since (contrary to AWS_REGION) we don't provide it in the following env arguments?

[simonrw]

Sorry I don't understand - what do you mean by "don't provide it in the following env arguments"? Do you mean in the README.md Configurations section? Or in the code? We do allow overriding the AWS_DEFAULT_REGION in the code.

[dominikschubert]

I mean

AWS_ENVAR_ALLOWLIST=AWS_REGION,AWS_DEFAULT_REGION AWS_REGION=eu-central-1 cdklocal ...
vs
AWS_ENVAR_ALLOWLIST=AWS_REGION,AWS_DEFAULT_REGION AWS_DEFAULT_REGION=eu-central-1 AWS_REGION=eu-central-1 cdklocal ...

[dominikschubert]

This is confusing me a bit as it seems to conflict with the PR description:

Raise an exception if the user specifies AWS_ENDPOINT_URL without AWS_ENDPOINT_URL_S3 as we cannot automatically determine a configuration that will work. We need to ensure that .s3. is in the domain for S3 requests. If the user customises the endpoint then it may not support subdomains (so we could not just inject .s3. in their URL, and we cannot use the same URL as our S3 detection won't work.