Add fetcher-default as a plugin

Question

Add fetcher-default as a plugin

balashashanka opened this issue 4 years ago · 4 comments

balashashanka commented 4 years ago

Currently the fetcher default is in package edu.usc.irds.sparkler.util, this issue is to move this as a plugin.

Answer 1 · 2020-05-05T10:53:05.000Z

Started working on this will be creating a pull request soon.

Answer 2 · 2020-05-05T22:35:18.000Z

Thanks.
we kept the default implementation in the sparkler-app itself for a few simplifications.

FYI, there are three fetcher plugins already if you want to test plugin loading (without having to develop a new one)https://github.com/USCDataScience/sparkler/tree/master/sparkler-plugins

That being said, all improvements are welcomed. just trying to make it easy for you.

Answer 3 · 2020-05-06T03:44:07.000Z

Hi I saw this is as one of the the TODO's in the code so thought it would be a good start.
But i could work on other things if this plugin is not in priority.

Answer 4 · 2020-05-06T19:23:42.000Z

You were facing an issue regarding deployment of plugins on sparkler cluster, right?
I suggest resolving that as a start.