Automagically generate tests to verify API responses or DB state.
pip install goldtest
You write goldtests as if you already have a "gold" file which contains the expected output of your program. This file will actually be generated by the test itself when you run it with the GOLDTEST_GEN
environment variable set. Under normal operation, goldtest will compare the output of your program to the gold file, and will fail the test if there are any differences.
import goldtest
class TestClass(goldtest.TestCase):
def test_testcase(self):
gt = goldtest.Goldtest(self)
resp = call_api() # function which returns a JSON-serializable object
gt.test('calling_api', resp)
Run the test with the GOLDTEST_GEN
environment variable set.
Example: GOLDTEST_GEN=1 python test_goldtest.py
Or, if you use pytest: GOLDTEST_GEN=1 py.test
Currently, it only works if you use SQLAlchemy. It has also only been tested with postgres.
# These 2 lines should not be in your test - they are often found in the model.
# They're only here to illustrate where db_metadata and db_engine come from.
engine = create_engine(...)
Base = declarative_base()
gt = goldtest.Goldtest(self,
db_metadata=Base.metadata,
db_engine=engine)
modify_db_state() # a function which modifies the state of the DB
gt.test_db('after_modification', tables=['users', 'accounts'])
Sometimes, parts of your output will be non-deterministic. Examples include timestamps, randomly generated tokens, etc. Don't worry, goldtest handles this!
It does so by running each test twice when generating a gold file. If a field changes in between runs, it will be replaced with "\*"
. This is invalid JSON, so it will never conflict with legitimate output. This "wildcard" lets goldtest know that anything is acceptable for this field when it is verifying output.