E3SM-Project/datasm

Isolate mapfile generation from publication for efficiency

Opened this issue · 0 comments

The warehouse "publish" workflow includes mapfile-generation and validation, movement to publication_root, mapfile-edit to accommodate path-changes, publishEsgf, and validateEsgf. Of these, the first (mapfile-generation) is very time-consuming, and should be able to be "launched" and completed weeks or months in advance of a publication authorization. We need to offer a "warehouse mapfiles" command that conducts only mapfile-generation and validation, and then halts. Thereafter, any "warehouse publish" command should be able to determine whether a suitable mapfile is already available, and proceed directly to the remaining publication steps (or else generate as needed).