does pretrained.cache_folder support self hosted S3A ?
cometta opened this issue · 2 comments
cometta commented
can pretrained.cache_folder save to self hosted S3A with custom endpoint or only support cloud based S3 ? when i run it keeps prompt me to enter value for spark.jsl.settings.aws.region which i don't need it. i need key for s3 custom endpoint
muhammetsnts commented
@maziyarpanahi can I ask for your support here please?
maziyarpanahi commented
- https://github.com/JohnSnowLabs/spark-nlp#spark-nlp-configuration
- Full working example : https://github.com/JohnSnowLabs/spark-nlp/blob/master/examples/util/Load_Model_From_S3.ipynb
- All the configs set in the SparkSession must be set, not some, not most, all of them are required. That's the only way to use a custom S3 bucket for cache_pretrained.
- We only support AWS S3 not any other S3A-like services The notebook shows how it works successfully
A typical S3 bucket (private or public) must have the path, key, secret, session key, region, etc.