[Feature idea] Automatic requirements.txt generation upon AML Experiment creation through Arcus
Opened this issue · 0 comments
Is your feature request related to a problem? Please describe.
Currently, when setting up ML experiment with arcus (below script), requirements.txt is not generated:
from arcus.azureml.environment.aml_environment import AzureMLEnvironment
work_env = AzureMLEnvironment.Create(config_file="../.azureml/config.json")
training_name = 'your_training_name'
trainer = work_env.start_experiment(training_name)
trainer.setup_training(training_name, overwrite=False)
Although we can simply manually create one, we may miss some crucial requirements for some of the azureml & arcus components such as the errors that come up linked to image processing (cv2). Issues have previously been opened regarding the import errors linked to images: #95 (comment)
Describe the solution you'd like
When executing the above script, arcus takes care of generating a requirements.txt file with at least the packages that are crucial to arcus & azureml to avoid errors (as mentioned above).
We could also extrapolate what packages are necessary in the requirements file by looking at what estimator type the user specified when executing a run:
trainer.start_training(training_name, estimator_type='tensorflow',
input_datasets = [dataset_name],
compute_target='your_compute_target', gpu_compute=True,
Typically here, we could automatically append to the requirements file: tensorflow, keras, scikit-learn, etc. However this would be a more "bonus" feature, the above is definitely a bigger priority.