wellcometrust/firebreak-sagemaker

Custom containers (with code and dependencies not working)

Opened this issue · 0 comments

The second example throws

Traceback (most recent call last):
  File "sagemaker/custom.py", line 38, in <module>
    args.role, args.image_uri)
  File "sagemaker/custom.py", line 26, in train_with_sagemaker
    es.fit({"training": data_path})
  File "/Users/nsorros/code/sagemaker/venv/lib/python3.7/site-packages/sagemaker/estimator.py", line 664, in fit
    self.latest_training_job = _TrainingJob.start_new(self, inputs, experiment_config)
  File "/Users/nsorros/code/sagemaker/venv/lib/python3.7/site-packages/sagemaker/estimator.py", line 1427, in start_new
    estimator.sagemaker_session.train(**train_args)
  File "/Users/nsorros/code/sagemaker/venv/lib/python3.7/site-packages/sagemaker/session.py", line 562, in train
    self.sagemaker_client.create_training_job(**train_request)
  File "/Users/nsorros/code/sagemaker/venv/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/nsorros/code/sagemaker/venv/lib/python3.7/site-packages/botocore/client.py", line 649, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/Users/nsorros/code/sagemaker/venv/lib/python3.7/site-packages/botocore/client.py", line 697, in _convert_to_request_dict
    api_params, operation_model)
  File "/Users/nsorros/code/sagemaker/venv/lib/python3.7/site-packages/botocore/validate.py", line 293, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Unknown parameter in InputDataConfig[0].DataSource: "FileDataSource", must be one of: S3DataSource, FileSystemDataSource

I was able to make it work, not 100% sure what is missing here as the same file paths are used in the other examples fine. Running in the cloud for the same example complains differently.