Plugins support for Import/Export resources
ITD27M01 opened this issue · 4 comments
Is your feature request related to a problem? Please describe.
Currently only GCP Cloud storage is supported for import and export of datapumps:
As a result such features are unusable in Enterprise environments with restricted networks and strict rules for data location.
Describe the solution you'd like
I suggest to change the "generic" Import/Export resources to use something like http/https urls and add ability to "load" plugins for different storages on different cloud providers or protocols (nfs, cifs or any other crazy things)
Hi Igor! This is a great suggestion. Do you have in mind any specific API/plugin/library that abstracts away storage for various cloud and on-prem providers?
Ash G.
@akinfermo Hi!
Last month I've been working with kaniko - a container image build tool inside a container or Kubernetes cluster. The interesting option is --context
where one defines path/url with the appropriate prefix to specify the location of your build context. It supports number of sources out of the box - local dir, targz, gcs, s3, git repo, etc... Inspired by such functionality, I would like to generalize the GcsPath
to something like ImportPath
and use the code to retrieve the prefix for appropriate backend storage:
func GetBuildContext(srcContext string, opts BuildOptions) (BuildContext, error) {
split := strings.SplitAfter(srcContext, "://")
if len(split) > 1 {
prefix := split[0]
context := split[1]
switch prefix {
case constants.GCSBuildContextPrefix:
return &GCS{context: context}, nil
case constants.S3BuildContextPrefix:
return &S3{context: context}, nil
case constants.LocalDirBuildContextPrefix:
return &Dir{context: context}, nil
case constants.GitBuildContextPrefix:
return &Git{context: context, opts: opts}, nil
case constants.HTTPSBuildContextPrefix:
if util.ValidAzureBlobStorageHost(srcContext) {
return &AzureBlob{context: srcContext}, nil
}
return &HTTPSTar{context: srcContext}, nil
case TarBuildContextPrefix:
return &Tar{context: context}, nil
}
}
return nil, errors.New("unknown build context prefix provided, please use one of the following: gs://, dir://, tar://, s3://, git://, https://")
}
Hi @ITD27M01
Your proposed approach sounds reasonable to me. Feel free to file a pull request with your proposed changes. Thank you!
Closing this due to inactivity.