Need help to understand the ids/numbers when upload for parallel jobs/builds.
qzhuyan opened this issue ยท 17 comments
I have a scenario that in one GitHub run I run cover for two languages thus I need to run two 'uploader' in sequential.
So the steps looks like this:
- run
make cover
and it generates cover data for language A and language B. - run coveralls github action (V2) to upload coverdata for language A.
- run another uploader command to upload the coverdata for language B.
- Mark the parallel run as finished.
I think the service_number
should be the same in steps 2, 3, and 4, correct?
Then I used different 'service_job_id' and different flag_name
in steps 2 and step 3.
The result is I always get the cover report in step 3 only.
example: https://coveralls.io/builds/60485505 (with debug=true)
I also see you have 'master' and 'main' branches, which branch of this action is recommended?
Hey!
Yes, you are right. service_number
should be the same for all parallel jobs, and for the one with parallel-finished: true
option. If you have all your coverage reports described in one workflow file - then you shouldn't care about providing service_number
- it is automatically fetched from ENVs.
I'd recommend using @v2
tag for your workflow because master
and main
behave a bit differently. master
branch and v1
tag uses node-coveralls
which is not being developed for a long time. main
branch and v2
tag use coverage-reporter
which is under active development and has a lot of bugs fixed. Probably no special configuration is required when you change master
to v2
.
Now I switched to V2 and from debugging info:
Two upload calls for two coverage report:
first one:
{
"service_branch": "test/william/prop-test",
"service_build_url": "https://github.com/qzhuyan/quic/actions/runs/5197308098",
"service_job_id": "build",
"service_job_url": "https://github.com/qzhuyan/quic/actions/runs/5197308098",
"service_name": "github",
"service_number": "5197308098",
"commit_sha": "e95a347600657a95f38f880badf969b2ddc0cc40",
"repo_name": "qzhuyan/quic",
"repo_token": "***",
"flag_name": "run-c-lcov",
"source_files": [
{
...}
"parallel": true,
second one:
Content-Type: application/json\r\n\r\n{\"commit_sha\":\"e95a347600657a95f38f880badf969b2ddc0cc40\",\"flag_name\":\"erlang\",\"parallel\":true,\"repo_token\":\"***\",\"service_job_id\":\"build\",\"service_job_number\":\"2\",\"service_name\":\"github\",\"service_number\":\"5197308098\",\"source_files\":[{\"coverage\":[null,null,nu ...
The service_number
are both 5197308098
The service_job_id
are both build
(feels wrong), maybe that is the reason one overrides the other?
The service_job_number
are different, one omit another is "2"
Both have parallel: true
I will tweak to use different service_job_id
and see what happens.
tested with different service_job_id
, one uploader use build
another uploader for use build2
, no luck.
any idea?
When using parallel: true
you have to close the build after all parallel jobs are reported.
- uses: coverallsapp/github-action@v2
with:
parallel-finished: true
Otherwise the build will remain open and won't calculate the overall coverage.
If this doesn't help, could you share your configuration, please? I will try to suggest a solution based on it.
the workflow is here:
https://github.com/qzhuyan/quic/blob/test/william/prop-test/.github/workflows/cover.yaml#L36-L61
We have two languages: C and Erlang
-
C:
We usecoverallsapp/github-action@v2
to upload cover report -
Erlang:
we use this rebar3 plugin https://github.com/qzhuyan/coveralls-erl to upload the report and for the API param they are defined here:
https://github.com/qzhuyan/quic/blob/test/william/prop-test/rebar.config.script#L6-L14
build log(unsure if you have the access):
https://github.com/qzhuyan/quic/actions/runs/5197440164
One build example:
https://coveralls.io/builds/60575827
Ok, it looks like Erlang plugin is a bit outdated and hasn't been updated for a long time.
Unfortunately, I am not familiar with Erlang, so I can't say what's exactly wrong here.
Can you say what you are expecting? Looks like builds don't receive Erlang coverage, right?
I just found it works before in March, see: https://coveralls.io/builds/57942310 but I don't have debug enabled and cannot see how API gets called.
What I expect is:
since I have two parallel (actually seq ) uploads (one for C and one for Erlang), I expect coveralls could merge them into one after parallel-finish is called.
The problem for me is the erlang upload seems to overwrite what C uploaded. The report in https://coveralls.io/builds/60575827
only shows the stuff from erlang upload. AND if I skip erlang upload I get report from C only.
No worry about erlang plugin, I have forked erlang plugin I could control how we call the API also with debugging info.
Since your backend is not opensource, I just want to get inputs from you on which params shall I use for doing this scenario.
some inputs will be very helpful like the following:
# first call API for C coverage
with:
service_number: 5197308098
service_job_id: "cover_C"
service_job_number: ""
parallel: true
## Second call for Erlang
with:
service_number: 5197308098
service_job_id: cover_erlang
service_job_number: "2"
parallel: true
third call to finish:
with:
service_number: 5197308098
parallel-finished: true
thanks!
If you want to merge them you need to provide a similar flag_name
parameter. Also service_number
and commit_sha
must be the same for both requests.
I can see that only one coverage is being uploaded (for C and LCOV), maybe there's an error when Erlang coverage is being uploaded. Unfortunately the response is not visible from logs, so I am not sure. Can you show the server response from Erlang uploader?
By the way, in case we get stuck, there is a converter of .coveradata to Cobertura XML format which is supported by coverage-reporter: https://github.com/covertool/covertool.
triggered new build: https://github.com/qzhuyan/quic/actions/runs/5197440164/jobs/9423205035
API call for uploading C part with following (using: coverallsapp/github-action@v2)
"service_branch": "test/william/prop-test",
2023-06-09T09:22:03.5717514Z "service_build_url": "https://github.com/qzhuyan/quic/actions/runs/5197440164",
2023-06-09T09:22:03.5718107Z "service_job_id": "build",
2023-06-09T09:22:03.5719170Z "service_job_url": "https://github.com/qzhuyan/quic/actions/runs/5197440164",
2023-06-09T09:22:03.5719698Z "service_name": "github",
2023-06-09T09:22:03.5720522Z "service_number": "5197440164",
2023-06-09T09:22:03.5721060Z "commit_sha": "bd403c8390cfa85f73bd810647d8c1a5dbf54a1c",
2023-06-09T09:22:03.5721806Z "repo_name": "qzhuyan/quic",
2023-06-09T09:22:03.5722466Z "repo_token": "***",
2023-06-09T09:22:03.5723259Z "flag_name": "run-c-lcov",
2023-06-09T09:22:03.6873324Z. "parallel": true,
2023-06-09T09:22:03.6873404Z "git": {
2023-06-09T09:22:03.6873836Z "branch": "test/william/prop-test",
2023-06-09T09:22:03.6873916Z "head": {
2023-06-09T09:22:03.6874060Z "id": "bd403c8390cfa85f73bd810647d8c1a5dbf54a1c",
2023-06-09T09:22:03.6874198Z "committer_email": "mscame@gmail.com",
2023-06-09T09:22:03.6874314Z "committer_name": "William Yang",
2023-06-09T09:22:03.6874438Z "author_email": "mscame@gmail.com",
2023-06-09T09:22:03.6874552Z "author_name": "William Yang",
2023-06-09T09:22:03.6874708Z "message": "ci(cover): service job id build2 for erlang"
2023-06-09T09:22:03.6874791Z }
2023-06-09T09:22:03.6874853Z },
2023-06-09T09:22:03.6875049Z "run_at": "2023-06-09T09:22:03+00:00"
2023-06-09T09:22:04.1424095Z โ API Response: {"message":"Coverage for parallel build uploaded","url":"https://coveralls.io/builds/60630675"}
now I get error code: 422 while I upload the erlang part.
with
"commit_sha":"bd403c8390cfa85f73bd810647d8c1a5dbf54a1c",
"flag_name":"erlang",
"parallel":true,
"repo_token":"***",
"service_job_id":"build2",
"service_job_number":"2",
"service_name":"github",
"service_number":"5197440164",
"source_files":[{"coverage":....
error I get...
422: "{\"message\":\"Can't add a job to a build that is already closed. Build 5197440164 is closed. See docs.coveralls.io/parallel-builds\",\"error\":true}"
Above is a rerun, the service_number is reused. Looks like it is not a good idea to use github run id as the service_number.
now I will trigger one new run and check the reply from coveralls while uploading Erlang.
ok with new build: https://github.com/qzhuyan/quic/actions/runs/5220552560/jobs/9423759253
C upload with: (using: coverallsapp/github-action@v2)
2023-06-09T09:51:04.1988221Z "service_branch": "test/william/prop-test",
2023-06-09T09:51:04.1988598Z "service_build_url": "https://github.com/qzhuyan/quic/actions/runs/5220552560",
2023-06-09T09:51:04.1988907Z "service_job_id": "build",
2023-06-09T09:51:04.1989204Z "service_job_url": "https://github.com/qzhuyan/quic/actions/runs/5220552560",
2023-06-09T09:51:04.1989505Z "service_name": "github",
2023-06-09T09:51:04.1989743Z "service_number": "5220552560",
2023-06-09T09:51:04.1990021Z "commit_sha": "a907bfb6b5b9be503c5ca2e3e40cfa032d534bf5",
2023-06-09T09:51:04.1990314Z "repo_name": "qzhuyan/quic",
2023-06-09T09:51:04.1990968Z "repo_token": "***",
2023-06-09T09:51:04.1991254Z "flag_name": "run-c-lcov",
2023-06-09T09:51:04.1991465Z "source_files": [....]
2023-06-09T09:51:04.2820100Z "parallel": true,
2023-06-09T09:51:04.2820178Z "git": {
2023-06-09T09:51:04.2820492Z "branch": "test/william/prop-test",
2023-06-09T09:51:04.2820563Z "head": {
2023-06-09T09:51:04.2820700Z "id": "a907bfb6b5b9be503c5ca2e3e40cfa032d534bf5",
2023-06-09T09:51:04.2820822Z "committer_email": "mscame@gmail.com",
2023-06-09T09:51:04.2820930Z "committer_name": "William Yang",
2023-06-09T09:51:04.2821045Z "author_email": "mscame@gmail.com",
2023-06-09T09:51:04.2821145Z "author_name": "William Yang",
2023-06-09T09:51:04.2821271Z "message": "chore: just trigger new build"
2023-06-09T09:51:04.2821337Z }
2023-06-09T09:51:04.2821393Z },
2023-06-09T09:51:04.2821558Z "run_at": "2023-06-09T09:51:04+00:00"
get resp:
2023-06-09T09:51:04.9292791Z โ
API Response: {"message":"Coverage for parallel build uploaded","url":"https://coveralls.io/builds/60631251"}
Erlang upload with:
{"commit_sha":"a907bfb6b5b9be503c5ca2e3e40cfa032d534bf5",
"flag_name":"erlang",
"parallel":true,
"repo_token":"***",
"service_job_id":"build2",
"service_job_number":"2",
"service_name":"github",
"service_number":"5220552560",
"source_files":[{"coverage":
...
And get resp:
{ok,{{"HTTP/1.1",200,"OK"},
2023-06-09T09:51:06.5695929Z [{"cache-control",
2023-06-09T09:51:06.5696408Z "max-age=0, private, must-revalidate"},
2023-06-09T09:51:06.5697045Z {"connection","keep-alive"},
2023-06-09T09:51:06.5697353Z {"date","Fri, 09 Jun 2023 09:51:06 GMT"},
2023-06-09T09:51:06.5697743Z {"etag","W/\"bfcb5e7503cecae0ea6dacc588038bb3\""},
2023-06-09T09:51:06.5698067Z {"server","cloudflare"},
2023-06-09T09:51:06.5699013Z {"content-length","95"},
2023-06-09T09:51:06.5699606Z {"content-type","application/json; charset=utf-8"},
2023-06-09T09:51:06.5700026Z {"status","200 OK"},
2023-06-09T09:51:06.5700525Z {"strict-transport-security",
2023-06-09T09:51:06.5701221Z "max-age=31536000; includeSubDomains"},
2023-06-09T09:51:06.5704393Z {"referrer-policy","strict-origin-when-cross-origin"},
2023-06-09T09:51:06.5704961Z {"x-permitted-cross-domain-policies","none"},
2023-06-09T09:51:06.5705555Z {"x-xss-protection","1; mode=block"},
2023-06-09T09:51:06.5708118Z {"x-request-id",
2023-06-09T09:51:06.5708590Z "0a167d78-d5f4-402e-af7d-5e08124b9657"},
2023-06-09T09:51:06.5709069Z {"x-download-options","noopen"},
2023-06-09T09:51:06.5709675Z {"x-frame-options","SAMEORIGIN"},
2023-06-09T09:51:06.5710443Z {"x-runtime","0.262724"},
2023-06-09T09:51:06.5710855Z {"x-content-type-options","nosniff"},
2023-06-09T09:51:06.5711206Z {"x-powered-by",
2023-06-09T09:51:06.5711514Z "Phusion Passenger(R) Enterprise 6.0.17"},
2023-06-09T09:51:06.5711877Z {"x-powered-by","cloud66"},
2023-06-09T09:51:06.5712243Z {"cf-cache-status","DYNAMIC"},
2023-06-09T09:51:06.5712563Z {"report-to",
2023-06-09T09:51:06.5713603Z "{\"endpoints\":[{\"url\":\"https:\\/\\/a.nel.cloudflare.com\\/report\\/v3?s=PPUKb4QP6wMODF%2Fibffq7dPvWMjkiegFFlC0K%2BKejOBo0mU7FfMMMMb8BGealyPYk61qjDUyL5aWwUKXRHiMwVJrV5qFITOO82pRrmQglWX3e5L0zzw%2F2IoN%2Fc%2F3wA%3D%3D\"}],\"group\":\"cf-nel\",\"max_age\":604800}"},
2023-06-09T09:51:06.5714711Z {"nel",
2023-06-09T09:51:06.5715144Z "{\"success_fraction\":0,\"report_to\":\"cf-nel\",\"max_age\":604800}"},
2023-06-09T09:51:06.5715560Z {"cf-ray","7d48795f3b02e781-DFW"},
2023-06-09T09:51:06.5715924Z {"alt-svc","h3=\":443\"; ma=86400"}],
2023-06-09T09:51:06.5716337Z "{\"message\":\"Coverage for parallel build uploaded\",\"url\":\"https://coveralls.io/builds/60631252\"}"}}
parallel finish call
2023-06-09T09:51:17.6762152Z โญ๏ธ Calling parallel done webhook: https://coveralls.io/webhook
2023-06-09T09:51:18.1549666Z ---
2023-06-09T09:51:18.1550589Z โ
API Response: {"done":true,"url":"https://coveralls.io/builds/60631252","jobs":1}
2023-06-09T09:51:18.1551373Z - ๐, Coveralls
I only get erlang report
Oh, I see, for some reason those reports go to the different builds (you can see that URLs differ in the responses). I need to debug this issue to find out if there's a problem on the backend side or not.
I think I found the issue and fixed it. The problem was that we weren't checking commit_sha
option, and Erlang integration doesn't provide git
options like coverage-reporter does.
You can start another build. I can suggest you the following change in your Erlang coverage plugin configuration:
- Remove
service_job_number
- it is not required - Don't edit
service_job_id
- now you append2
to the end, but this is not required
it works now! thanks!
https://coveralls.io/builds/60633707
https://github.com/qzhuyan/quic/actions/runs/5221556893/jobs/9425963829
but I got another issue:
https://coveralls.io/jobs/123018755/source_files/9867975771
"the commit SHA "0f536fed6d318fbc28ce978f39d2276cce789b7a" was not found in your repository, so the file cannot be loaded. This may be because you posted from a local development environment, or your CI created an ephemeral commit."
It must be a temporary issue related to Github API limits. Could you try one more time?
solved ! apprecated! ๐