testcontainers/testcontainers-python

Bug: tests are run inconsistently in CI pipeline

max-pfeiffer opened this issue · 2 comments

max-pfeiffer commented

Describe the bug

Maybe not a real bug but more an issue I came across when I looked at how our tests are run in CI pipeline:

So looking at pyproject.toml I guess we want to support Python versions from 3.9 - 4.0, so our pipeline needs to reflect this.

I also see that we do not pin the versions of our Ubuntu runners. I would say it's a better practice to pin version to get predictable CI results.

Also I am scratching my head what purpose that make file still serves there. I see no need to use it in the pipeline jobs.

Also we use coverage, but are not doing anything with that test coverage data.

I could spend some time fixing this up. But I would be very grateful for some feedback and thoughts. Not so sure if I miss something here.

totallyzen commented

hey @max-pfeiffer thanks for the insights!

Personal note: I monitor the testcontainers slack more often than the issues, don't have the time to monitor both, sorry about that - please do come and discuss on slack, (you can also raise the issue here, but it may take me and others longer to act)

Back to your issue:

  • community modules
    • maybe you are right, should remove the python-version from that and hard-code it on 3.11
    • since we only write pure python in here, it's probably a better matrix option to change runners
    • the multi-arch runners has been on my mind but zero energy to do anything with it
  • makefile
    • it's useful for auto-generating entrypoints for the pipeline to use - this also keeps things in line between local and CI runs (zero difference!)
    • make install can be incredibly useful to set up everything according to our standards (see #460)
    • I've had #460 ongoing but got halted by shitty work weeks and health issues. please come and comment on that one to improve the makefile together (I'm not in favour of removing it just yet)
max-pfeiffer commented

@totallyzen Ok, so I will create a PR addressing these inconsistency issues and pin runner versions.

I just went through #460 and left some comments and suggestions.

I haven't used slack in ages. But I will check on that channel now.