spectraphilic/reflexible

refactoring of workflow for reflexible

Opened this issue · 0 comments

Currently the workflow for reflexible is as a post processing tool for FLEXPART:

import reflexible as rf
H = rf.Header('/path/to/directory')
...
do some post-processing with the Header

When flexpart runs, it initially reads a single file called pathnames with the following structure:

options/
OUTPUT/
/
/AVAILABLE_ECMWF_EI_fields_global
============================================

The first line points to an "options" directory that contains further input files (e.g. RELEASES, CONTROL, etc.). The second line points to the output directory that would contain the header output netcdf or fortran binary files.

This proposal for a new standard workflow suggests pointing to the pathnames file to initiate reflexible. It is not required (e.g. we should maintain the possibility to do H = rf.Header(... but, it could be an improvement is folks work flow started with the pathnams file.

As we are now implementing a read_releases class, this could be done as R = rf.Releases('/path/to/release_file').

However, integrating this into a full reflexible workflow might look something like:

import reflexible as rf
fprun = rf.Flexpart('path/to/pathnames')

H = fprun.Header
R = fprun.Release
C = fprun.Control
 ...

In doing this, we should evaluate to some extent what information on the header (which now comes entirely from the output files), should be taken from input files (e.g. CONTROL file). And whether, in fact, we should conduct some consistency tests and raise a warning if the output file is not consistent with the input file...

The initiation could, for example, probably create most of the Header from the CONTROL file (see legacy.pflexible for a read_control example). Then, while instantiating, it could check if there is any output created, or if the directory is empty (or maybe doesn't even exist) and provide a warning:

[1]import reflexible as rf
[2]fprun = rf.Flexpart('/path/to/pathnames')
"Reading Flexpart Configuration"
"Warning, empty [non existent] output directory"

[3]H = fprun.Header
[4]R = fprun.Release