Batch Array Jobs do not receive AWS_BATCH_JOB_ARRAY_INDEX
Closed this issue · 1 comments
mkochco commented
Description
According to AWS docs when a Batch job is submitted with arrayProperties the resulting array jobs should be executed with an environment that contains AWS_BATCH_JOB_ARRAY_INDEX.
Localstack's Batch support doesn't appear to behave this way.
Repro;
Follow the localstack batch instructions on setting up a batch compute environment and a job queue for that environment.
Create the following job definition;
$ awslocal batch register-job-definition \
--job-definition-name test-array-job \
--type container \
--container-properties '{"image":"busybox","vcpus":1,"memory":128,"command":["env"]}'
Submit a batch array job using that job def
$ awslocal batch submit-job \
--job-name array-job \
--job-queue myqueue \
--job-definition test-array-job \
--array-properties size=2
Check that 2 jobs were created
$ awslocal batch list-jobs --job-queue myqueue
Inspect the logs for the jobs
$ awslocal logs tail /aws/batch/job --log-stream-name-prefix test-array-job
2024-09-01T22:24:53.246557+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-09-01T22:24:53.246591+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 HOSTNAME=fc4fc0199318
2024-09-01T22:24:53.246593+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 AWS_BATCH_JOB_ID=d502baaa-cd14-4d8f-a166-344edf382195:0
2024-09-01T22:24:53.246595+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 AWS_ACCESS_KEY_ID=test
2024-09-01T22:24:53.246596+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 AWS_SECRET_ACCESS_KEY=test
2024-09-01T22:24:53.246598+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 LOCALSTACK_HOSTNAME=172.17.0.2
2024-09-01T22:24:53.246599+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 MOTO_HOST=http://172.17.0.2
2024-09-01T22:24:53.246600+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 MOTO_PORT=5000
2024-09-01T22:24:53.246602+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 MOTO_HTTP_ENDPOINT=http://172.17.0.2:5000
2024-09-01T22:24:53.246603+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:0 HOME=/root
2024-09-01T22:24:53.246623+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-09-01T22:24:53.246695+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 HOSTNAME=cc652528357e
2024-09-01T22:24:53.246702+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 AWS_BATCH_JOB_ID=d502baaa-cd14-4d8f-a166-344edf382195:1
2024-09-01T22:24:53.246705+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 AWS_ACCESS_KEY_ID=test
2024-09-01T22:24:53.246708+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 AWS_SECRET_ACCESS_KEY=test
2024-09-01T22:24:53.246711+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 LOCALSTACK_HOSTNAME=172.17.0.2
2024-09-01T22:24:53.246714+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 MOTO_HOST=http://172.17.0.2
2024-09-01T22:24:53.246717+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 MOTO_PORT=5000
2024-09-01T22:24:53.246720+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 MOTO_HTTP_ENDPOINT=http://172.17.0.2:5000
2024-09-01T22:24:53.246724+00:00 test-array-job/default/d502baaa-cd14-4d8f-a166-344edf382195:1 HOME=/root
Actual Result:
No AWS_BATCH_JOB_ARRAY_INDEX in the log output
Expected result:
Logs contain AWS_BATCH_JOB_ARRAY_INDEX
mkochco commented
Submitted against wrong project