Test notebooks run as part of CI checks
nathancooperjones opened this issue · 1 comments
Is your feature request related to a problem? Please describe.
Not a problem, rather something that would be nice to eventually have.
Currently we have tutorial notebooks and a quickstart notebook in the tutorials directory that we do not test as we make changes to the library.
Describe the solution you'd like
Ideally, we would have a part of the CI checks for each PR that runs all Jupyter notebooks and only allows merging if the notebooks run without error.
Describe alternatives you've considered
Worst case scenario, we can just add something to the PR checklist ensuring that notebooks still run, but that's not as nice I think.
Any additional information?
I think this might be possible by having the test first export the notebook as a python file programmatically, then have the test run the Python file in its entirety. We don't really care about output, just that we finish without error.
Looks like a tool to enable this has been created: nbmake 🎉