Tests against 'num_ref' property of LiftWing data are failing
Opened this issue · 3 comments
In recent builds — such as https://github.com/WikiEducationFoundation/WikiEduDashboard/actions/runs/11304166699/job/31442185321#step:14:4022 — it looks like all the tests that use the num_ref data from the LiftWing API are failing. Perhaps the API format has changed?
Determine the cause of these failing tests and make any necessary updates.
@ragesoss can i take this
Findings from the Issue
After analyzing the response from the get_revision_data and get_number_of_references_from_revision_ids method (which pulls data from the LiftWing and ReferenceCounter APIs), no structural changes were observed in the API responses. However, multiple tests failed or returned nil values in the latest build, and the test run time extended to 2 hours 16 minutes—about four times longer than current build time.
Possible Reasons for the Build Failure:
VCR Cassette Issue:
- The VCR cassette may have recorded a failed or incomplete API response, possibly due to an unstable internet connection. * The test logic assigns a nil value to num_ref when this occurs, leading to failures in subsequent runs.
Inconsistent Test Behavior:
When run individually, all tests passed, suggesting either:
- Steps were skipped during the main build.
- Cached data from previous successful runs was reused, leading to discrepancies.
Incorrect VCR Values:
- For the revisions where num_ref failed, the expected values were present in the API, indicating that the VCR cassette might have stored incorrect data during the initial recording.
Possible solution to this, will be to
Clear and re-record the VCR cassettes to capture fresh API responses, ensuring no erroneous data from previous builds is reused, especially when test failures or inconsistencies arise