Run margin generation within the main import pipeline
hombit opened this issue · 0 comments
hombit commented
Feature request
I believe that margin generation should become more implicit and happen by default for every hipscat-import
. I propose falling changes:
- Generate margins with default 10'' margin_threshold within the "main" importing pipeline. This should be the default, however we still may allow a separate margin generation pipeline to a) generate more caches, b) split the pipeline for large/complicated imports.
- Turn off fine filtering by default
Before submitting
Please check the following:
- I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
- I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
- If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.