Add Logic to Handle Pre-Existing Docker Images in AML Environment Creation Script (Hard)

[GrannyProgramming]

What is the Issue?

I am using the script create_environments.py to manage and create AML environments based on configuration files found in environments folder. Currently, the script is effective in checking and creating Conda environments against provided user configurations. However, when it comes to handling Docker images, the script falls short and cannot verify pre-existing Docker images against the provided configuration.

What would the solution look like?

I propose to enhance the script by adding logic to verify pre-existing Docker images against the user's configuration. This will enable us to check whether a Docker image that matches the user's configuration already exists in the AML environment, thereby preventing unnecessary image creation and saving resources. The steps would include:

Extract metadata from Docker image: We should be able to extract the metadata from a Docker image by either using Docker Python SDK or Azure ML SDK. Docker Python SDK can allow us to interact with Docker images like reading Dockerfile or inspecting a Docker image to get its metadata.

Compare Docker image metadata: Once we have the metadata of the Docker image, we should compare it with the Docker image configuration provided by the user. If there is a match, that means the Docker image already exists and we don't need to create a new environment.

Updating the function create_or_update_docker_environment: The function should be updated to include the Docker metadata extraction and comparison logic.

Any alternative solutions?
An alternative could be manually checking the Docker images against the configuration file. However, automating this process within the script would be more efficient and less prone to error.

Resources and Relevant links
There are no relevant material online I could find