arcus-azure/arcus.azureml

[Feature idea] Automatic requirements.txt generation upon AML Experiment creation through Arcus

Opened this issue · 0 comments

Is your feature request related to a problem? Please describe.
Currently, when setting up ML experiment with arcus (below script), requirements.txt is not generated:

from arcus.azureml.environment.aml_environment import AzureMLEnvironment
work_env = AzureMLEnvironment.Create(config_file="../.azureml/config.json")
training_name = 'your_training_name'
trainer = work_env.start_experiment(training_name)
trainer.setup_training(training_name, overwrite=False)

Although we can simply manually create one, we may miss some crucial requirements for some of the azureml & arcus components such as the errors that come up linked to image processing (cv2). Issues have previously been opened regarding the import errors linked to images: #95 (comment)

Describe the solution you'd like
When executing the above script, arcus takes care of generating a requirements.txt file with at least the packages that are crucial to arcus & azureml to avoid errors (as mentioned above).
We could also extrapolate what packages are necessary in the requirements file by looking at what estimator type the user specified when executing a run:

trainer.start_training(training_name, estimator_type='tensorflow', 
                       input_datasets = [dataset_name], 
                       compute_target='your_compute_target', gpu_compute=True, 

Typically here, we could automatically append to the requirements file: tensorflow, keras, scikit-learn, etc. However this would be a more "bonus" feature, the above is definitely a bigger priority.