This repo represents the topsApp Product Generation Executable (PGE) for the TopsApp portion of the standard product pipeline generating interferograms from Sentinel-1.
This code was primarily written by Mohammed Karim and David Bekaert. The original code lives in the ariamh repo (along with other PGEs).
This code was reorganized so that the same python script could be used for the following two pipelines:
coseismicstandard-product
The script uses information from the job-spec of the PGE to perform the processing relevant to that pipeline (note there are two pairs of job-spec and hysds-io files in the docker directory corresponding to each pipelines PGE). The job-spec metadata is recorded within the _context.json per general HySDS operations.
To ensure jobs are not erroneously run using datasets across pipelines, we also check the machine tags of the ifg-cfg datasets used to create the _context.json to ensure the input datasets were slinged by the correct pipeline.
The best way to build the image for this PGE is via the Dockerfile. Specifically from the terminal, navigate to this repository and then build the docker image using:
docker build -f docker/Dockerfile -t standard_product .
See the testing section for more details on how to test and interact with the code. Note that the context of the docker build must be this repo.
April, 2021: An update to elasticssearch using pip. There are warnings/Errors because this is incompatible with mozart and hysds-core. Fortunately, I don't use these tools within this PGE.
TopsApp takes in SLC data from Sentinel-1 and runs the TopsApp from ISCE2 automatically. See the linked notebook for a description of the algorithm and how to use this ISCE2 processing outside of the PGE.
Use the end-to-end tests expounded in the tests directory. Please see the tests/readme.md for more details.
Warning: each end-to-end test directory can take anywhere from 30 GB (for restricted areas of interest) to 130 GB (for those run on the full area).
Warning: permissions of a mounted volume within a docker container may change the permissions of local files. I ignore permission changes for git tracking per this thread.
- v1.0.0
- Reorganized Repo from here
- v2.0.4
- Reorganized Repo from here
- Fork this repo
- Create your feature branch (
git checkout -b feature/fooBar) - Commit your changes (
git commit -am 'Add some fooBar') - Push to the branch (
git push origin feature/fooBar) - Create a new Pull Request
Please put issues in Issues page in this repo so we can track needed feature requets.