IBM/FfDL

Parameterize DIND scripts

fplk opened this issue · 0 comments

fplk commented

While originally only intended as a quick help to setup FfDL's dependencies, the DIND scripts have become more widely used than anticipated. Thus, we should probably overhaul and parameterize them. Here are a few notes for improvement:

  • https://github.com/IBM/FfDL/blob/master/bin/dind_scripts/create_user.sh should have an environment variable for the username, so it can be adapted in one place

  • https://github.com/IBM/FfDL/blob/master/bin/dind_scripts/experimental_master.sh line 2 and 24 should use $USER, so this is not tied to the user being ffdlr

  • We should make sure $GOPATH is used consistently, so users who do not use ~/go can adapt it

  • The scripts should probably check more rigorously whether they were successful and potentially create reports, so tracing issues during setup becomes easier.

  • The current deploy-plugin target has side effects, i.e. it not only pulls and deploys the S3 driver plugin, but it also sets up a hostmount PV, which is not the standard for development and should probably be done in an independent task. [Also: Isn't that an NFS replacement, whereas the rest is about S3?]

  • The test-submit Makefile target will not work when used against existing S3 buckets, since it uses placeholder names for the buckets and endpoint. We either need to replace those like we replace the username and password or we need to halt the experimental_master.sh script and tell the user to adapt the etc/examples/tf-model/manifest.yml file before running make test-submit

  • Overall, we might want to turn the scripts into a full installer that queries for username, target directory etc., we should also add macOS support and potentially allow for more customization (e.g. use existing container registry rather than local registry)