kennethreitz/records

Flaws of current `.travis.yml` test suite

vlcinsky opened this issue · 2 comments

vlcinsky commented

Current test suite as defined in travis.yml and Makefile has couple of problems:

  • tests only python 2.7 and 3.6
  • difficult to run locally for multiple python versions
  • Makefile require make, which is not always available under Windows making test setup more difficult
  • use of pipenv brings problems:
    • test is slowed down
    • current Pipfileinstalls dependencies, which shall be installed with records package, effectively hiding possible problems in setup.py requirements (e.g. setup.py omitting important dependency, which shall be installed by it).
    • is is not clear, where gets pytest installed

Proposal for fix is:

  • do not use pipenv for testing (Pipfile.lock is not used, requirements shall be defined by setup.py so default section in Pipfile shall be empty or only with -e .) Vulnerability to pypi python package updates can be considered as positive feature (detecting real life cases, which shall work well)
  • refactor .travis.yml to tox
vlcinsky commented

61b2b68 is fixing issue with setup.py containing utf-8 encoded chars, which were failing when building the package under python 2.7.

Older version of .travis was not detecting this error as it probably did not build the package and used it in development mode.

vlcinsky commented

See #141 Testing under Python 2.6 revealed:

  • sqlalchemy must be of older version
  • python installed prints deprecation warning about not supported version.