Specification of path for update_state in ips-framework
parkjm opened this issue · 15 comments
@elwasif merge_current_plasma_state calls an executable "update_state" in the user's path. "update_state" has to be different, depending on workflows. It would be very helpful if the framework support for users to specific the binary of update_state. For example, CESOL needs many different update_state for FASTRAN, C2, SOLPS, etc. in a single workflow that calls nested workflows. This also helps to manage environment where we maintain "update_state" even for the PPPL plasma state-based workflows.
This whole infrastructure was designed to allow concurrent updating of the common plasma state from multiple components running concurrently, and this way of doing things is very PPPL plasma state specific. If we want to extend this beyond that one use case, I think we need to think more as to how this should be done. Supporting "genera;" concurrent access to the plasma state involves more than just user-specified update_state code.
We can leave how to implement update_state to be consistent with con-current runs for the workflow developers. What would be the involvement of the framework other than calling "update_state" to make it consistent with con-current runs?
I'm not sure I understand, if we're not using it in a concurrent situation, then one should just use update_state instead of merge_state anyway.
I mean that there are other workflows like IPS-C2 and IPS-SOLPS-ITER that could benefit from concurrent run. The developers of IPS-C2 and IPS-SOLPS-ITER should be able to implement their own update_state. Another example is for use OMAS as an option for the plasma state.
The "update_state" used in the framework is expected to be an executable from PPPL plasma state that takes in a "partial plasma state" filled by a component and inserts it into the "current global combined state". So if two components concurrently generate non-overlapping partial states, they can merge their updates into the global state without conflict (gentlemen agreement using different "modules" in the PPPL PS - the framework doesn't perform any conflict detection). So if we want to extend this and make it more "generic", we need to come up with exactly what will the framework do and what will be the semantics of this new "merge state" service call.
The framework does not do any conflict detection for PPPL's "update_state". Why do we want to do more if we want to use other generic "update_state"?
Technically, if the general update_state executable has the same command line arguments as the PPPL's one:
update_state -input <target_state> -updates <partital_state>
are we ready to go (with a new capability of specifying the update_state executable path)?
The framework does not do any conflict detection for PPPL's "update_state". Why do we want to do more if we want to use other generic "update_state"?
Technically, if the general update_state executable has the same command line arguments as the PPPL's one:
update_state -input <target_state> -updates <partital_state>are we ready to go (with a new capability of specifying the update_state executable path)?
If the command line interface we currently have works for you, then yes. One option is to add an optional argument to the merge_state() call to specify the path to the binary you want to use (you can populate that from a config section entry if you want). Also target_state is hard-wired to the file defined in CURRENT_STATE. Keep in mind that in addition to updating the shared plasma state in the common state directory, the updated file is copied over to the calling component's working directory. So after this call you have the updated state locally (including any possible updates from other components that are running concurrently if they made a call to merge_state).
One option is to add an optional argument to the merge_state() call to specify the path to the binary you want to use (you can populate that from a config section entry if you want)
Perfect.
Do these workflows have a common data structure for which multiple components need to update different data elements?
Yes, for example, IPS-C2 has its own plasma state (fortran namelist and necdf for 2D variables) and calls 2D transport solver for many impurities simultaneously. Each C2 call involves update of the C2 plasma state only for single impurity.
CESOL uses IPS-FASTRAN, IPS-C2, IPS-SOLPS and IPS-EPED as a component, each having their own plasma state. We can call these compound workflows simultaneously.
How is OMAS going? Is it an option for plasma state yet?
In progress for IPS-FASTRAN. The next step is for 2D SOL.
So one thing to remember, if each component is updating a separate FILE from the plasma state files, then you don't need to use merge_state at all. You can specify this subset of files in the component's configuration section, and only those files will be impacted by any call to stage_state() or update_state()
Sure. Please update the Cori installation when the optional argument is available.
Can you give it a try now.