Error #01: Needed resources are used by other tasks.
ishaulov opened this issue · 11 comments
Hi! Need help with publishing packages!
Packages are assembled on Jenkins, and then they are further uploaded and published to the repository using the aptly api:
curl -f -X POST -F file=@"libson-1.11.6248_amd64.deb" http://repos.local/deb/api/files/uploaded-files?_async=true -u repouser:repopassword
curl -f -X POST http://repos.local/deb/api/repos/debrepo/file/uploaded-files/libson-1.11.6248_amd64.deb?_async=true -u repouser:repopassword
curl -f -X PUT http://repos.local/deb/api/publish/:./debrepo?_async=true -H 'Content-Type: application/json' --data '{"Signing":{"Batch":true,"GpgKey":"gpgkey"}}' -u repouser:repopassword
curl -f -X PUT http://repos.local/deb/api/publish/:./ubuntu-20.04?_async=true -H 'Content-Type: application/json' --data '{"Signing":{"Batch":true,"GpgKey":"gpgkey"}}' -u repouser:repopassword
curl -f -X PUT http://repos.local/deb/api/publish/:./mint?_async=true -H 'Content-Type: application/json' --data '{"Signing":{"Batch":true,"GpgKey":"gpgkey"}}' -u repouser:repopassword
curl -f -X PUT http://repos.local/deb/api/publish/:./ubuntu-22.04??_async=true -H 'Content-Type: application/json' --data '{"Signing":{"Batch":true,"GpgKey":"gpgkey"}}' -u repouser:repopassword
The problem is that there may be parallel builds, downloads, and publications at the same time, and I get the following error:
aptly — "Error #1: Needed resources are used by other tasks."
curl — (22) The requested URL returned error: 409 Conflict
{"level":"info","time":"2024-04-18T09:15:42+05:00","message":"Executing task asynchronously"}
{"level":"info","remote":"172.27.0.4","method":"POST","path":"/api/repos/debrepo/file/uploaded-files/libson-1.11.6248_amd64.deb?_async=true","protocol":"HTTP/1.0","code":"202","latency":"166.954µs","agent":"curl/7.58.0","time":"2024-04-18T09:15:42+05:00"}
{"level":"info","time":"2024-04-18T09:15:42+05:00","message":"Executing task asynchronously"}
{"level":"warn","remote":"172.27.0.4","method":"PUT","path":"/api/publish/:./debrepo?_async=true","protocol":"HTTP/1.0","code":"409","latency":"4.423243ms","agent":"curl/7.58.0","time":"2024-04-18T09:15:42+05:00","message":"Error #01: Needed resources are used by other tasks."}
{"level":"info","time":"2024-04-18T09:15:42+05:00","message":"Executing task asynchronously"}
{"level":"warn","remote":"172.27.0.4","method":"PUT","path":"/api/publish/:./ubuntu-20.04?_async=true","protocol":"HTTP/1.0","code":"409","latency":"4.386ms","agent":"curl/7.58.0","time":"2024-04-18T09:15:42+05:00","message":"Error #01: Needed resources are used by other tasks."}
{"level":"info","time":"2024-04-18T09:15:42+05:00","message":"Executing task asynchronously"}
{"level":"warn","remote":"172.27.0.4","method":"PUT","path":"/api/publish/:./mint?_async=true","protocol":"HTTP/1.0","code":"409","latency":"4.258301ms","agent":"curl/7.58.0","time":"2024-04-18T09:15:42+05:00","message":"Error #01: Needed resources are used by other tasks."}
{"level":"info","time":"2024-04-18T09:15:42+05:00","message":"Executing task asynchronously"}
{"level":"warn","remote":"172.27.0.4","method":"PUT","path":"/api/publish/:./ubuntu-22.04?_async=true","protocol":"HTTP/1.0","code":"409","latency":"4.272133ms","agent":"curl/7.58.0","time":"2024-04-18T09:15:42+05:00","message":"Error #01: Needed resources are used by other tasks."}
Have I tried using this version:
#1125 (comment)
and using ?_async=true, but the error persists.
I tried to build aptly from the master branch, errors persist.
What else can I do, or maybe I'm doing something wrong?
Hi, thanks for reporting !
yes, this might be fixed with #1125, which is currently being rebased on master.
the ?_async=true will make the API calls return a task_id. the caller should wait for the task to complete, before issuing a next api call. this can be done with GET /tasks/:id/wait
maybe you could try this ?
Thanks for the reply!
Yes, I can. But let me clarify: do I understand correctly that I need to write a shell handler that will process each curl request, take the ID and put it into GET /tasks/:ID/wait ?
yes, something like this:
curl http://repos.local/api/tasks/1/wait
Hello, @neolynx!
I apologize for the delay in replying, it took a while to check everything out.
I've added /tasks/:ID/wait
in my shell script, but I still get this error when building in parallel on Jenkins:
[GIN] 2024/05/21 - 13:41:39 | 200 | 83.949261ms | 172.19.0.2 | GET "/api/tasks/9/wait"
2024/05/21 13:41:39 Executing task asynchronously
[GIN] 2024/05/21 - 13:41:39 | 202 | 2.403509ms | 172.19.0.2 | PUT "/api/publish/:./ubuntu-20.04?_async=true"
[GIN] 2024/05/21 - 13:41:39 | 200 | 87.470772ms | 172.19.0.2 | GET "/api/tasks/10/wait"
[GIN] 2024/05/21 - 13:41:47 | 200 | 310.719834ms | 172.19.0.2 | POST "/api/files/uploaded-files?_async=true"
2024/05/21 13:41:47 Executing task asynchronously
[GIN] 2024/05/21 - 13:41:47 | 202 | 115.25µs | 172.19.0.2 | POST "/api/repos/libs/file/uploaded-files/libs-generator.87.6316_amd64.deb?_async=true"
[GIN] 2024/05/21 - 13:41:47 | 200 | 307.632802ms | 172.19.0.2 | POST "/api/files/uploaded-files?_async=true"
2024/05/21 13:41:47 Executing task asynchronously
[GIN] 2024/05/21 - 13:41:47 | 409 | 116.276µs | 172.19.0.2 | POST "/api/repos/libs/file/uploaded-files/libs-generator.87.6317_amd64.deb?_async=true"
Error #01: Needed resources are used by other tasks.
Aptly version 1.5.0+ds1-1+b4 .
This is stdout of shell script with set -x
:
+ echo 'Moving deb package to repository...'
Moving deb package to repository...
++ curl -f -X POST 'http://repos.local/deb/api/repos/libs/file/uploaded-files//libs-generator.87.6317_amd64.deb?_async=true' -u repo-api:aGF#EQLr
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 53 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 409
And do I understand correctly that I can't get ID and STATE on this request?
+ echo 'Uploading deb package: libsdeb///libs-generator.87.6317_amd64.deb'
Uploading deb package: libsdeb///libs-generator.87.6317_amd64.deb
++ curl -s -f -X POST -F file=@libsdeb///libs-generator.87.6317_amd64.deb 'http://repos.local/deb/api/files/uploaded-files?_async=true'
+ upload_response='["uploaded-files//libs-generator.87.6317_amd64.deb"]'
+ '[' 0 -ne 0 ']'
+ echo '["uploaded-files//libs-generator.87.6317_amd64.deb"]'
+ json_pp
[
"uploaded-files//libs-generator.87.6317_amd64.deb"
]
And is it correct that on GET /tasks/:ID/wait
I should get STATE=2
? Is there any actual documentation on this function somewhere?