DRfC not working in AWS 'opt in' regions
Closed this issue · 5 comments
AWS has a list of original regions which are all enabled by default, and then a list of 'opt-in' regions which a user must enable. Some of these opt-in regions have good prices for spot instances and therefore it's desirable to be able to use them.
It appears that DRFC doesn't work in AWS 'opt in regions', RLCoach image exits: -
The suggestion from my research is this is because for opt-in regions the region must be specified when making a request, even to S3, and I assume it isn't in the docker containers today, hence above error.
I tested on an Ubuntu linux machine, first request you can see fails, second request with region specified works: -
Perhaps this could be solved by adding this to the early part of the docker image when running on AWS, becuase this will set the region and then doesn't require you to set it with each command?: -
export AWS_DEFAULT_REGION=$(curl http://169.254.169.254/latest/meta-data/placement/region)
this would then set the region to the region the instance is running in having queried the metadata service.
I tried fixing it the simple way in DRfC's docker file. Can you check if this is sufficient? (app_region branch)
The DR_AWS_APP_REGION must be set in system.env
Tested successfully, as long as you set AWS_DEFAULT_REGION=eu-south-1 in your terminal window (else some fo the dr-* commands don't work).
Screenshot of running training in Milan: -
Reopened. As discussed Lars dr-download-custom-files doesn't work without the region being set in the terminal. I imagine other commands that need to interact with the S3 bucket from within the terminal rather than the docker image would also have the same issue.
@MarkRoss-Eviden - added the environment variable; could you test now. (app-region branch)
Tested in Milan (one of the opt in regions) with setting DR_AWS_APP_REGION=eu-south-1 in system.env.
Works great so ready to merge @larsll : -
dr-start-training
