Investigating the possibility of only having one solution vector in systems
YaqiWang opened this issue · 3 comments
Currently, System builds two solution vectors, solution and current_local_solution always. Conceptually, we do not need this and this can create unnecessary memory usage, which can be a problem in extreme matrix-free large simulations. @roystgnr mentioned that this could be a legacy with old PETSc and things could have been changed. Thus, it is possible to remove one solution vector. Typically, the non-ghosted solution vector should be removed but if a system is only used for auxiliary purpose, a solution vector with only local DoFs could be sufficient. The impact of this change could be quite large because lots of depending codes in MOOSE and MOOSE based application assume two vectors. But hopefully the refactoring can be smooth because the two vectors are in synchronization. Tag @lindsayad
I think this will be tricky to make backwards compatible since solution and current_local_solution are public members, not hidden behind accessors. One thought is that System::update() could be changed so that it doesn't do the localize step from solution -> current_local_solution... then current_local_solution could still live in System, but only certain subclasses of System would actually initialize/use it, perhaps by setting some needs_current_local_solution flag.
They're public members, but they're members wrapped in unique_ptr, so one could be left unallocated, if we were careful about not actually trying to dereference it in library code or application code when this feature is active.
But ... no, that still wouldn't work. Even in code that doesn't conceptually need both a solver-compatible solution and a ghosting-enabled current_local_solution, the standard way to work is to write to solution (as the canonical data), update(), then read from current_local_solution (to get the complete data on local elements). We'd need to add new setter/getter methods and transition to those before we could even begin reworking the underlying data structures, wouldn't we?
I've already warned Yaqi that I wouldn't be looking at this before October at the earliest, but now I'm wondering whether even then it'd worth bulldozing everything, unless we were able to go even further and do something like merging the two vectors entirely.
conceptually it would be nice to only have one vector