Suggestion: Change data source file structure
Itaykal opened this issue · 2 comments
Itaykal commented
TL;DR - Make each data source a package instead of a module.
Changing data_sources folder to contain foldersinstead of a file for each source.
It will make the code less "monolithic" without creating a meess, you could implement your own clients/wrappers for those clients which will make the code much more easy to maintain. add files for testing, etc etc.
Have a look at CloudQuery's solution. (although it's an extremely different product for a different use case..)
I guess it's better to do that sooner than later ;/
rishi003 commented
I was looking at the data_sources folder too, and I completely agree. A structure would make it more maintainable overall. I would also recommend using a mono repo tooling for the entire project.