woodpecker-ci/woodpecker

New repos list triggers rate limiting

Opened this issue · 3 comments

zc-devs commented

Component

server, web-ui

Describe the bug

Testing new repo list, sometimes I get errors : max connections reached: 10.
Having 7 repos:

  1. for each repo there are requests to pipeline endpoint
  2. somehow they are doubled, so I have 14 requests to pipeline endpoint

Steps to reproduce

  1. Woodpecker + Gitea
  2. Create some repos in Gitea, 10 for example
  3. Enable them in Woodpecker
  4. Open repositories list
  5. Optionally, see : max connections reached: 10 errors
  6. Open browser console, network
  7. See, that UI tries to load pipeline info for each repo twice
    Screenshot 2024-11-21 133824

Expected behavior

  1. One request for last pipelines info for all repos like pipelines/last?repo_ids=[1,2,3,4,5]
  2. Or send last pipeline info with repo info in response to repos list
  3. Else at least fix those doubled requests
  4. In addition to 3, reduce the pagination page size

System Info

WP `next-350082cd19`, Gitea `1.22.3`, Postgres `16`, Kubernetes `v1.30.6+k3s1`

Additional context

#4386 (comment)

Validations

  • Read the docs.
  • Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
  • Checked that the bug isn't fixed in the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]
anbraten commented

Loading it twice should be fixed.

Do you know where the max connection reached error came from. Never saw that before. Do you have anything in the console? Incase of http2 the browser should share the connection shouldn't it.

zc-devs commented

Yes, that's from Traefik. It's HTTP 429 Too Many Requests with response body max connections reached: 10. Seems, in the test cluster I have a lower value of 10. That is why I wrote Optionally, see ... in 5.

Despite my configuration, the point is still valid, IMO. There is a request per repo. How many repos per pagination-page? 50? So there will be at least 50+1 requests, if I have > 50 repos. That is not good to me. If my proposals 1 and 2 do not suit you, what is about reducing pagination-page size to... let's say 10? (Added this as proposal 4).

PS: feel free to rename the issue.

qwerty287 commented

That's actually a problem. The repos list is not paginated at all currently.

  • 👍1