CI is silently reporting failures (run is marked as passing)
lhstrh opened this issue · 7 comments
See https://github.com/mikepenz/release-changelog-builder-action/actions/runs/5905658059 for instance. Also, tests are skipped in the main branch, which makes is a bit difficult to figure out whether it is actually passing the tests or not.
@lhstrh I assume you refer to the errors shown as annotations?
Tests are executed for PRs -> and also when we merge from dev -> main
Currently v4 is practically still a release candidate so I have not done the dev -> main merge.
However in all those scenarios tests are passing.
You can also run the tests within a GitHub codespace
I looked into that some more.
There were some test cases from the very beginning which were testing empty changelogs which unfortunately used a tag which wasn't existing. Adapted those.
Other cases the output is expected. E.g. we have a test-case where we expect the tag not to exist. and similar.
See the updates output here: https://github.com/mikepenz/release-changelog-builder-action/actions/runs/5974510275
Does this answer your concerns?
Thanks for looking into it. Yes, this looks better. That being said, because the tests have your handle hard-coded in them, I'm not able to run them on my own... See https://github.com/lhstrh/release-changelog-builder-action/actions/runs/5981870608/job/16230305596?pr=1 for example.
Those errors look strange. The only test I'd expect to fail is:
which uses a personal token.However all the others should work. That being said the failure you sent are compile errors and not test failures.
I suppose that may be due to the changes you made to the code and APIs.
For example: lhstrh@d396f4f#diff-dffdf4fda345500572a96793a7bb8ee4e361c7cd7e412dff123890ea3ba80f85R19
If you run the tests on the original state, they shall pass.
OK, I'll dig a bit deeper when I get a chance. Let me first fix the moving submodules problem so I can get my release published :-)
Following up on this ticket, is there any additional input I can provide? (or are we otherwise good to close it?)
Oh, I never realized that besides your main branch called develop there was another branch called main where your stable code lives. Usually people call their main branch main... Anyhow, let's close this issue.
I'll make an effort to make the tests pass in my fork and then get back to you. If you'd like to adopt the "submodules" and/or "summary" feature that I developed, then I think I might need your help merging your upstream changes...