openedx/wg-devops

Fix lms & cms container environments to run tests without modification

Opened this issue · 9 comments

kdmccormick commented

Context

Here are the instructions on how to run tests in Tutor: https://docs.tutor.overhang.io/dev.html#running-edx-platform-unit-tests

Notice that:

  1. the lms container must be used, even for cms tests
  2. one needs to fiddle with a few different environment variables in order to run pytest on each root folder (lms, cms, common, openedx, xmodule).

Both of these limitations are awkward and potential stumbling blocks for developers.

Note on lms vs cms tests

edx-platform runs in one of two modes: LMS (the learning/instruction interface & APIs) and CMS (Studio and authoring APIs). Some code is specific to LMS or CMS, and some code is shared between the two. Both LMS and CMS have their own Django settings files.

In the edx-platform source tree...

  • The tests under ./lms/ are LMS-specific and should only be run with LMS settings.
  • The tests under ./cms/ are CMS-specific and should only be run with CMS settings.
  • The tests under ./openedx/, ./common/, and ./xmodule/ are "shared". That, the code could be used by both LMS and CMS, so we want to be able to run unit tests both using LMS settings and CMS settings, and the tests should pass under both contexts.
    • Exceptions: Some unit tests in these directories are LMS-specific or CMS-specific. Those tests are marked with decorators @skip_unless_lms and @skip_unless_cms, respectively.

edx-platform's PR checks ensure that the above is always true on the master branch.

Acceptance

Tweak edx-platform's test setup and/or Tutor's cms+lms dev container environments such that, without having to change any environment variables:

  • One can run all LMS-specific tests and shared tests in the lms container using LMS Django settings.
  • One can run all CMS-specific tests and shared tests in the cms container using CMS Django settings.

In other words, these should just work:

tutor dev run lms pytest lms/ xmodule/ common/ openedx/
tutor dev run cms pytest cms/ xmodule/ common/ openedx/

Notes

kdmccormick commented

@bradenmacdonald can you just leave a comment here? GH won't let me make you an assignee until you do.

kdmccormick commented

@bradenmacdonald I'm going to start poking at this as I have time this week, and I'll post here if I learn anything. I don't know if your situation makes you more or less likely to work on this 😛 but if you do, let me know what you find!

bradenmacdonald commented

Ok sounds good @kdmccormick. Please let me know what you end up finding either way :)

kdmccormick commented

I've got a couple WIP PRs:

Haven't fully confirmed that they solve the problem, but I'm hopeful...

kdmccormick commented

The two PRs above are ready for feedback.

kdmccormick commented

Going to come back to this issue after the conference. See overhangio/tutor#648 (comment)

kdmccormick commented

unfortunately I haven't been able to dig into this more, so I'm unassigning myself for now.

@bradenmacdonald were you still hoping to look into this, or should I unassign you?

bradenmacdonald commented

@kdmccormick I don't think I'll have time anytime soon unfortunately.

  • 👍1