aws/aws-cdk

(aws-lambda-python: PythonFunction): unable to use bundling in BitBucket

TRANTANKHOA opened this issue · 6 comments

TRANTANKHOA commented

I want to specify a custom location for the output volume of the PythonFunction docker build instead of the default.

Use Case

When I build my PythonFunction inside bitbucket pipeline it result in the following errors:

Successfully built f9a2edaf9e2d
Successfully tagged cdk-027978ada45c27f70df7fa8b7d578bed26985c21721a65193aba99e73aee35e3:latest
Bundling asset TestStateMachine/TestStateMachine/TaskMonitoringLambda/Code/Stage...
docker: Error response from daemon: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories.
See 'docker run --help'.
FAIL test/mamcdk.test.ts (26.586 s)
  CDK Application
    ✕ encountered a declaration exception (9 ms)
  ● CDK Application › encountered a declaration exception
    Failed to bundle asset TestStateMachine/TestStateMachine/TaskMonitoringLambda/Code/Stage, bundle output is located at /tmp/cdk.out6iRb2m/asset.d942a7d32e9693e439c345510f27daa13c2af102da78a30659f334a62cefd8f6-error: Error: docker exited with status 125

The gist of it is docker: Error response from daemon: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories.

Proposed Solution

let user specify custom output volume location

This is a 🚀 Feature Request

eladb commented

@TRANTANKHOA I am a bit confused here - is this about the source of the volume (i.e. in the host) or the target of the volume (in the container). Feels like BitBucket should only care about the source, no?

Can you run with --verbose --debug and upload your log so I can see the docker command? I suspect the problem is that the source of the volume is in a temp directory which is not under the bitbucket clone directory.

TRANTANKHOA commented

thanks @eladb for your questions, just to provide you with more of the background.

In my cdk app I have this snippet

new PythonFunction(this, 'TaskMonitoringLambda', {
    entry: path.join(__dirname, '..', 'lambda'),
    index: 'failed_task_notification.py',
    handler: 'push',
    runtime: Runtime.PYTHON_3_7,
    retryAttempts: this.node.tryGetContext('max_retries'),
    memorySize: 128,
    functionName: `${machineID}TaskMonitoringLambda`,
    assetHashType: AssetHashType.SOURCE, // see https://github.com/aws/aws-cdk/pull/12984
    environment: {
        'FAILED_TASK_TOPIC': failedTaskTopic.topicArn,
        'REGION': this.node.tryGetContext('region'),
        'ENV': env
    }
})

then when I run npm run test inside bitbucket pipeline container the resulting log is below

2021-05-04 08:22:18 INFO     Running: npm run build
> mamcdk@0.1.0 build /opt/atlassian/pipelines/agent/build
> tsc
2021-05-04 08:22:26 INFO     Success code 0
2021-05-04 08:22:26 INFO     Running: npm run test
> mamcdk@0.1.0 test /opt/atlassian/pipelines/agent/build
> jest
Sending build context to Docker daemon  2.048kB
Step 1/4 : ARG IMAGE=public.ecr.aws/sam/build-python3.7
Step 2/4 : FROM $IMAGE
latest: Pulling from sam/build-python3.7
fa1d3331d853: Pulling fs layer
...

Digest: sha256:9a02c80a8b16bdd3a63d3dd2215299f277debbd47aed83decbf7523b36850cb1
Status: Downloaded newer image for public.ecr.aws/sam/build-python3.7:latest
 ---> ddb84d69c69b
Step 3/4 : RUN yum -q list installed rsync &>/dev/null || yum install -y rsync
 ---> Running in 20292b31ed7d
Removing intermediate container 20292b31ed7d
 ---> 4cdbbe0924b1
Step 4/4 : CMD [ "python" ]
 ---> Running in 97ec842bd31b
Removing intermediate container 97ec842bd31b
 ---> f9a2edaf9e2d
Successfully built f9a2edaf9e2d
Successfully tagged cdk-027978ada45c27f70df7fa8b7d578bed26985c21721a65193aba99e73aee35e3:latest
Bundling asset TestStateMachine/TestStateMachine/TaskMonitoringLambda/Code/Stage...
docker: Error response from daemon: authorization denied by plugin pipelines: -v only supports $BITBUCKET_CLONE_DIR and its subdirectories.
See 'docker run --help'.
FAIL test/mamcdk.test.ts (26.586 s)
  CDK Application
    ✕ encountered a declaration exception (9 ms)
  ● CDK Application › encountered a declaration exception
    Failed to bundle asset TestStateMachine/TestStateMachine/TaskMonitoringLambda/Code/Stage, bundle output is located at /tmp/cdk.out6iRb2m/asset.d942a7d32e9693e439c345510f27daa13c2af102da78a30659f334a62cefd8f6-error: Error: docker exited with status 125
      36 |             displayName: `${machineID}FailedTaskTopic`
      37 |         })
    > 38 |         this.failedTaskNotification = new PythonFunction(this, 'TaskMonitoringLambda', {
         |                                       ^
      39 |             entry: path.join(__dirname, '..', 'lambda'),
      40 |             index: 'failed_task_notification.py',
      41 |             handler: 'push',
      at AssetStaging.bundle (node_modules/@aws-cdk/core/lib/asset-staging.ts:472:13)
      at AssetStaging.stageByBundling (node_modules/@aws-cdk/core/lib/asset-staging.ts:330:10)
      at stageThisAsset (node_modules/@aws-cdk/core/lib/asset-staging.ts:196:35)
      at Cache.obtain (node_modules/@aws-cdk/core/lib/private/cache.ts:24:13)
      at new AssetStaging (node_modules/@aws-cdk/core/lib/asset-staging.ts:221:44)
      at new Asset (node_modules/@aws-cdk/aws-s3-assets/lib/asset.ts:127:21)
      at AssetCode.bind (node_modules/@aws-cdk/aws-lambda/lib/code.ts:277:20)
      at new Function (node_modules/@aws-cdk/aws-lambda/lib/function.ts:583:29)
      at new PythonFunction (node_modules/@aws-cdk/aws-lambda-python/lib/function.ts:106:5)
      at new StepFunctionMonitoring (lib/stepFunctionMonitoring.ts:38:39)
      at new MamRiskStateMachineStack (lib/mamRiskStateMachineStack.ts:95:27)
      at Suite.Object.<anonymous>.describe (test/mamcdk.test.ts:20:38)
      at Object.<anonymous> (test/mamcdk.test.ts:11:1)
 › 1 snapshot obsolete.
   • CDK Application State Machine Created 1
Snapshot Summary
 › 1 snapshot obsolete from 1 test suite. To remove it, run `npm test -- -u`.
   ↳ test/mamcdk.test.ts
       • CDK Application State Machine Created 1
Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   1 obsolete, 0 total
Time:        27.33 s
Ran all test suites.

From which I noted that the PythonFunction's bundle output is located at /tmp/cdk.out* while I can also confirm from Atlassian documentation that its bitbucket containers are set up so that -v only supports $BITBUCKET_CLONE_DIR and its subdirectories.

Since I'm not running docker command directly but it's triggered through CDK internal implementation, I'm just guessing that CDK is sending the bundle output and declaring a docker volume to the /temp/* directory which is not under the git repo root directory hence the error.

So I think it's an issue with compatibility between CDK and Bitbucket. Hope this helps!

jogold commented

This only happens during tests right?

Are you creating an App in your test? You should be able to specify an outDir for this App to avoid this error. Something like:

const app = new App({ outDir: path.join(process.env.BITBUCKET_CLONE_DIR, 'test') });
const stack = new Stack(app, 'TestStack');
  • 👍2
eladb commented

Yes, this is likely the issue - the default outDir for App is a temporary directory, which will not be under the BITBUCKET_CLONE_DIR.

Closing this for now. @TRANTANKHOA let me know that you are unblocked.

github-actions commented

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

TRANTANKHOA commented

thanks @jogold , just the thing I need!