Bug in tutorial with references
createyourpersonalaccount opened this issue · 4 comments
In https://reframe-hpc.readthedocs.io/en/stable/tutorial.html#adding-performance-references there's a definition for myhost:baseline
. But as I've discovered after testing, it is more like mysystem:mypartition
.
This becomes more confusing since later in the tutorial (see https://reframe-hpc.readthedocs.io/en/stable/tutorial.html#id11) a baseline.py configuration is shown where a hostname myhost
is defined for the mytutorialsys
system and an environment with name baseline
. The correct reference for that would be mytutorialsys:default
.
There are three related issues overall with the presentation:
- The issue I've described above; it is
mysystem:mypartition
for references and needs to be fixed in the tutorial. - The order of presentation should be reversed; we should first discuss configuration files and then references.
- The
valid_prog_environs
is not clearly explained. It took me a while to figure out how to request two features, (withvalid_prog_environs = ['+A +B']
.) In retrospect, it is explained in https://reframe-hpc.readthedocs.io/en/stable/regression_test_api.html#reframe.core.pipeline.RegressionTest.valid_systems but it would be nice if the example showed more than just one feature.
If you agree with these issues I can fix them.
In https://reframe-hpc.readthedocs.io/en/stable/tutorial.html#adding-performance-references there's a definition for myhost:baseline. But as I've discovered after testing, it is more like mysystem:mypartition.
Yes, this need fixing. I would probably go with generic:default
which is what we are using and explain its meaning pointing to the explanation of systems/environment in the following section.
The issue I've described above; it is mysystem:mypartition for references and needs to be fixed in the tutorial.
Agreed.
The order of presentation should be reversed; we should first discuss configuration files and then references.
I'm not sure I'm in favor of this. Because if you want to run local run-only tests you can even go without dealing with the configuration. Configuration it's a whole chapter of its own and if it comes first, I think the point about how writing tests will be missed. The idea is to introduce the necessary parts of the configuration as the reader progresses with the tutorial.
The valid_prog_environs is not clearly explained. It took me a while to figure out how to request two features, (with valid_prog_environs = ['+A +B'].) In retrospect, it is explained in https://reframe-hpc.readthedocs.io/en/stable/regression_test_api.html#reframe.core.pipeline.RegressionTest.valid_systems but it would be nice if the example showed more than just one feature.
Agreed, but I would rather make the reader aware of this possibility and point them to the reference for more examples.
@vkarak I'll take a look on Monday to submit a patch.
I'll push it back a bit until the end of the week.
I'll push it back a bit until the end of the week.
No rush, thanks!