openfisca/openfisca-core

Improve package's doctests

Opened this issue · 1 comments

bonjourmauko commented

Hi there!

I really enjoy OpenFisca, but I recently encountered an issue.

Here is what I did:

pytest openfisca_core openfisca_web_api --maxfail 0 --continue-on-collection-errors --quiet

Here is what I expected to happen:

XXX passed, X xfailed, XXX warnings in X.XXs

Here is what actually happened:

========================================================================================= short test summary info ==========================================================================================
FAILED openfisca_core/commons/formulas.py::openfisca_core.commons.formulas.apply_thresholds
FAILED openfisca_core/commons/formulas.py::openfisca_core.commons.formulas.switch
FAILED openfisca_core/holders/holder.py::openfisca_core.holders.holder.Holder.get_memory_usage
...
ERROR openfisca_core/scripts/measure_numpy_condition_notations.py::test_switch_select

Context:

Python provides doctests to help keep a testable documentation.

They provide some benefits:

  • They help maintainers and contributors to spot bugs and improvement opportunities.
  • They helps reusers to understand the code quickly
  • They help separating contexts (unit, functional, ...)
  • They help produce the official doc

Complications:

Today the code coverage by doctests is mild, they are not run, and most of them are broken.

Some observed consequences:

  • The code is in overall hard to grasp without a consequent time investment, thus hard to contribute to
  • The code is hard to test unitarily due to its current design
  • There are then very few unit tests
  • The documentation is was broken

Proposal:

Fix, complete, and run doctests systematically!

TODO

  • #1033
  • #1167
  • #1034
  • #1051
  • #923
  • 6. Improve openfisca_core.parameters doctests
  • #1138
  • 8. Improve openfisca_core.populations doctests
  • 9. Improve openfisca_core.projectors doctests
  • 10. Improve openfisca_core.reforms doctests
  • 11. Improve openfisca_core.simulation_builder doctests
  • 12. Improve openfisca_core.simulations doctests
  • 13. Improve openfisca_core.taxbenefitsystems doctests
  • 14. Improve openfisca_core.taxscales doctests
  • 15. Improve openfisca_core.tracers doctests
  • 16. Improve openfisca_core.variables doctests
  • 17. Improve openfisca_web_api.loader doctests
MattiSG commented

Thanks @maukoquiroga for providing a bit more background to this impressive series of PRs 🙂

Could you explain what “to work” and “it didn't” means for pytest? Did you not get failures when you expected to get them? Did the command simply not run?

  • 👍1