alexanderjfink/miner

Build a basic plugin architecture

Opened this issue · 0 comments

A basic plugin architecture should cover at least the map processing tools -- download, unpack, install, and clean up. A plugin, for example, could do of more ethnic to every downloaded file, could replace how the files are downloaded. It would also need to be able to add a CLI option for a plugin. Maybe even the database driver use could be managed by plugins?

The first potential plugin is in Issue #14. I'd like to do the downloading of data through the Tor network for anonymous and encrypted data. It would be an optional to use CLI option that would vastly slow down the downloading (its gonna take a while to download big data sets through Tor) but would make it secure and anonymous. For people downloading datasets that are in some way political, it might be best that it remain anonymous. Since it isn't part of the core mission of this project, I think it would be better to have it plug in but be separate.

I may try to use this basic plugin architecture:
http://martyalchin.com/2008/jan/10/simple-plugin-framework/

Or this one: http://yapsy.sourceforge.net/

More information that I found about how to build these (since I've never built a plugin architecture before) was here: http://stackoverflow.com/questions/932069/building-a-minimal-plugin-architecture-in-python