Testing with MODFLOW 6 examples

Question

Testing with MODFLOW 6 examples

Closed this issue 2 years ago · 10 comments

Task

I tried to use xmipy with all examples that come with the standard installation of MODFLOW 6.3.0.

Setup

Essentially, I ran:

mf6 = XmiWrapper(dll_path)
mf6.initialize(nam_file)
mf6.finalize()

for all models in the examples directory.

This is the script run_xmipy_model.py to run one model:

from pathlib import Path
import os
import sys

from xmipy import XmiWrapper
from xmipy.utils import cd

DLL_PATH = os.getenv('DLL_PATH')

def run_model(dll_path=DLL_PATH):
    nam_file = sys.argv[1]
    model_path = str(Path(nam_file).parent)
    with cd(model_path):
        try:
            print(f'    START {nam_file}')
            mf6 = XmiWrapper(dll_path)
            print(f'    INSTANCE')
            mf6.initialize(nam_file)
            print(f'    INITIALIZED')
            mf6.finalize()
            print(f'    FINALIZED')
            print(f'    GOOD {nam_file}')
        except:
            print(f'    BAD {nam_file}')
            raise



if __name__ == '__main__':
    run_model()

This is the script test_m6_examples.py to run all models:

from pathlib import Path
import os
from subprocess import run


MF6_EXAMPLE_PATH = os.getenv('MF6_EXAMPLE_PATH')


def run_model(nam_file):
    run(['python', 'run_xmipy_model.py', str(nam_file)])


def test_run_all(example_dir=Path(MF6_EXAMPLE_PATH)):
    root = example_dir.resolve()
    for model_path in root.iterdir():
        nam_file = model_path / 'mfsim.nam'
        print(f'RUNNING {nam_file}')
        run_model(nam_file=nam_file)
        print(f'DONE {nam_file}')
        print()


if __name__ == '__main__':
    test_run_all()

To reproduce:

Set environmental variables: MF6_EXAMPLE_PATH to absolute path of the example directory such as C:\Users\xxx\mf6.3.0\examples and DLL_PATH to absolute path to libmf6.dll
Run test_m6_examples.py.

Result

A suscessful run should have an output like this:

RUNNING ...\mf6examples\ex-gwf-u1disv\mfsim.nam
    START ...\mf6examples\ex-gwf-u1disv\mfsim.nam
    INSTANCE
    INITIALIZED
    FINALIZED
    GOOD ...\mf6examples\ex-gwf-u1disv\mfsim.nam
DONE  ...\mf6examples\ex-gwf-u1disv\mfsim.nam

Most models produce this output.

Some models seem to not initialize the model:

RUNNING  ...\mf6examples\ex-gwt-mt3dms-p02f\mfsim.nam
    START  ...\mf6examples\ex-gwt-mt3dms-p02f\mfsim.nam
    INSTANCE
DONE  ...\mf6examples\ex-gwt-mt3dms-p02f\mfsim.nam

Other models seem to have serious problems:

RUNNING  ...\mf6examples\ex-gwt-mt3dms-p04a\mfsim.nam
    START  ...\mf6examples\ex-gwt-mt3dms-p04a\mfsim.nam
    INSTANCE
    INITIALIZED
forrtl: severe (153): allocatable array or pointer is not allocated
Image              PC                Routine            Line        Source
libmf6.dll         00007FFE4059BA68  Unknown               Unknown  Unknown
libmf6.dll         00007FFE401E920B  Unknown               Unknown  Unknown
libmf6.dll         00007FFE40364D08  Unknown               Unknown  Unknown
libmf6.dll         00007FFE404A5BBF  Unknown               Unknown  Unknown
libmf6.dll         00007FFE402B6D42  Unknown               Unknown  Unknown
libmf6.dll         00007FFE401CD8EA  Unknown               Unknown  Unknown
libmf6.dll         00007FFE401C0129  Unknown               Unknown  Unknown
ffi-8.dll          00007FFEDA7E4541  Unknown               Unknown  Unknown
ffi-8.dll          00007FFEDA7E4332  Unknown               Unknown  Unknown
ffi-8.dll          00007FFEDA7E4212  Unknown               Unknown  Unknown
_ctypes.pyd        00007FFEDEB5B3BF  Unknown               Unknown  Unknown
_ctypes.pyd        00007FFEDEB5BDAB  Unknown               Unknown  Unknown
_ctypes.pyd        00007FFEDEB56908  Unknown               Unknown  Unknown
python310.dll      00007FFEB215BA87  Unknown               Unknown  Unknown
python310.dll      00007FFEB226FF02  Unknown               Unknown  Unknown
python310.dll      00007FFEB226AE00  Unknown               Unknown  Unknown
python310.dll      00007FFEB226E37F  Unknown               Unknown  Unknown
python310.dll      00007FFEB215BB8E  Unknown               Unknown  Unknown
python310.dll      00007FFEB2267741  Unknown               Unknown  Unknown
python310.dll      00007FFEB226FD02  Unknown               Unknown  Unknown
python310.dll      00007FFEB226C003  Unknown               Unknown  Unknown
python310.dll      00007FFEB226E37F  Unknown               Unknown  Unknown
python310.dll      00007FFEB215BB8E  Unknown               Unknown  Unknown
python310.dll      00007FFEB2267741  Unknown               Unknown  Unknown
python310.dll      00007FFEB226FD02  Unknown               Unknown  Unknown
python310.dll      00007FFEB226C003  Unknown               Unknown  Unknown
python310.dll      00007FFEB226E37F  Unknown               Unknown  Unknown
python310.dll      00007FFEB215BB8E  Unknown               Unknown  Unknown
python310.dll      00007FFEB2267741  Unknown               Unknown  Unknown
python310.dll      00007FFEB226FD02  Unknown               Unknown  Unknown
python310.dll      00007FFEB226BA5D  Unknown               Unknown  Unknown
python310.dll      00007FFEB226E37F  Unknown               Unknown  Unknown
python310.dll      00007FFEB22E0FE1  Unknown               Unknown  Unknown
python310.dll      00007FFEB22E10C8  Unknown               Unknown  Unknown
python310.dll      00007FFEB22E0CC8  Unknown               Unknown  Unknown
python310.dll      00007FFEB22DEAEB  Unknown               Unknown  Unknown
python310.dll      00007FFEB20DB8CA  Unknown               Unknown  Unknown
python310.dll      00007FFEB20DC377  Unknown               Unknown  Unknown
python310.dll      00007FFEB20DD243  Unknown               Unknown  Unknown
python310.dll      00007FFEB20DD2B6  Unknown               Unknown  Unknown
python.exe         00007FF66FCE1490  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FFEEBD27034  Unknown               Unknown  Unknown
ntdll.dll          00007FFEEC1A2651  Unknown               Unknown  Unknown
DONE  ...\mf6examples\ex-gwt-mt3dms-p04a\mfsim.nam

Answer 1 · 2022-08-15T10:37:24.000Z

Thanks @pya, @mjr-deltares will have a look at this.

CC: @jdhughes-usgs, @langevin-usgs

Answer 2 · 2022-08-18T12:01:01.000Z

Hi @pya, thanks for doing such a careful scan of our API functionality. I reproduced your findings and I think I have a clear idea what is going on. You call Initialize and Finalize, but before we were making use of the API, it was not uncommon to allocate variables not until the first Update of the program (c.f. the call to xt3d_ar(...) in dsp_ad(...)). Since you are skipping Update in your tests, the program tries to deallocate something that has not yet been allocated, resulting in the exceptions that you report.

We will discuss in the MODFLOW development team if calling Initialize and Finalize without Updates in between is going to be supported or not.

Is there any reason why you would be particularly interested in that? (other than making for more time-efficient testing like this one :-))

Answer 3 · 2022-08-18T15:35:57.000Z

Hi @mjr-deltares I had the same though shortly after I posted this. I updated my script as shown below, calling .update() at each time step. Still the same result, i.e. errors.

import sys

from pymf6.mf6 import MF6


def run_model():
    nam_file=sys.argv[1]
    try:
        mf6 = MF6(nam_file=nam_file)
        current_time = mf6.get_current_time()
        end_time = mf6.get_end_time()

        while current_time < end_time:
            mf6.update()
            current_time = mf6.get_current_time()
        mf6.finalize()
        print(f'    GOOD {nam_file}')
    except:
        print(f'    BAD {nam_file}')
        raise



if __name__ == '__main__':
    run_model()

What else should I change to make it work?

As for my interest in this, it is always good to see if it works for the base case before doing any modifications. The MODFLOW examples make real good base cases.

Answer 4 · 2022-08-18T15:49:26.000Z

Hey @pya, the concept you are pursuing should work. I don't see mf6.initialize() in your most recent run_model function. Is that the problem?

Answer 5 · 2022-08-18T18:10:08.000Z

Hey @langevin-usgs, it works. :) Edited the wrong file. I had two files with similar names. All examples work. Takes much longer to run since I update at each time step. Here is the script:

from pathlib import Path
import os
import sys

from xmipy import XmiWrapper
from xmipy.utils import cd

DLL_PATH = os.getenv('DLL_PATH')

def run_model(dll_path=DLL_PATH):
    nam_file = sys.argv[1]
    model_path = str(Path(nam_file).parent)
    with cd(model_path):
        try:
            print(f'    START {nam_file}')
            mf6 = XmiWrapper(dll_path)
            print(f'    INSTANCE')
            mf6.initialize(nam_file)
            print(f'    INITIALIZED')
            current_time = mf6.get_current_time()
            end_time = mf6.get_end_time()

            while current_time < end_time:
                mf6.update()
            current_time = mf6.get_current_time()
            mf6.finalize()
            print(f'    FINALIZED')
            print(f'    GOOD {nam_file}')
        except:
            print(f'     BAD {nam_file}')
            raise



if __name__ == '__main__':
    run_model()

Much better, but this one still crashes:

ex-gwtgwt-mt3dms-p10

These stop at INSTANCE, i.e. never show INITIALIZED:

ex-gwf-csub-p02c
ex-gwt-hecht-mendez-b
ex-gwt-hecht-mendez-c
ex-gwt-keating
ex-gwt-moc3d-p01a
ex-gwt-moc3d-p01b
ex-gwt-moc3d-p01c
ex-gwt-moc3d-p01d
ex-gwt-moc3d-p02
ex-gwt-moc3d-p02tg
ex-gwt-mt3dms-p02a
ex-gwt-mt3dms-p02b
ex-gwt-mt3dms-p02c
ex-gwt-mt3dms-p02d
ex-gwt-mt3dms-p02e
ex-gwt-mt3dms-p02f
ex-gwt-mt3dsupp631
ex-gwt-mt3dsupp632a
ex-gwt-mt3dsupp632b
ex-gwt-mt3dsupp632c
ex-gwt-mt3dsupp82
ex-gwt-prudic2004t2

Why is this?

Answer 6 · 2022-08-18T20:04:16.000Z

We've made a few changes to ex-gwtgwt-mt3dms-p10 so it's important that the dylib/dll is consistent with the version of this problem. I suspect one or the other is out of date.

Not sure about the other problems. Will need to look further.

Answer 7 · 2022-08-22T08:00:27.000Z

Hi @pya, have you looked at the output for those model runs? Particularly the content of the file mfsim.stout would be interesting.

Answer 8 · 2022-08-22T09:15:36.000Z

Not yet. @mjr-deltares What do think about turning my little scripts into proper tests with better, automatic reporting? I can make a PR. What project would this belong to, to xmipy or more upstream, i.e. to MODFLOW6 itself?

Answer 9 · 2022-08-22T09:31:43.000Z

Hi @pya, I think these tests could be a valuable addition. And upstream makes more sense to me, as they are really MODFLOW integration tests. I will discuss within the MODFLOW dev team (we had similar ambitions before, it's just one of those things on the list...).

(And yes, your PRs are always welcome, just know that we have all the pieces of code available for running such tests, so it might not be needed ;-))

Answer 10 · 2023-01-10T08:23:30.000Z

I think we can close this one now as the focus of the discussion has moved away from the scope of xmipy.