Tests rely on API responses that can change in the future
Closed this issue · 4 comments
https://github.com/circleguard/circlecore/blob/release/tests/test.py#L133-L141 or all test_user_slice Tests expect data which we can't guarantee to be correct (Basically this test will most likely fail in the future)
That's entirely correct, but unless we choose a map that's been 4 modded and capped spinners, that will be true for any map. And more generally it's impossible to test our api interactions are working properly without using something that can change over time (ie a user's top scores).
Unless you have a better proposal, this us the best we can do currently. Updating test cases once every couple months at most isn't the end of the world.
I assume the correct solution would be to simulate api responses. We don't want to check if the api (or ossapi for that matter) is working, we are testing if our logic stuff is working... Though that might be too much work to implement, so I guess we will stick to Tests that may break at any moment.
Faking api responses is indeed the proper way to go about this (api test cases would be covered by ossapi itself, although it has none atm). But that's quite a lot of work—I'm not even sure how we would implement that cleanly—for little benefit.
going to close this, tests have only needed to be updated once or twice over the past two years. I don't think this is a big enough deal to fix