new limit in SA file
Opened this issue · 7 comments
8月开始发现 SA 的限制更多了.
之前一直是千兆独服跑 pt/bt 上传. 配置了自动 rotate SA. 一天换一个账户的 100个 SA. 一周或一个月重复. 现在这些 SA 都遇到了 403 限制.
请问有没有其他人遇到了相似的情况.
9月15日，也出现了100个SA，直接全部被403限制的问题，现在只能换成普通google账户来上传
Im experiencing this issue as well.
Its not a gclone issue looks like something that Google has changed,
I have over 51k service accounts in rotation across over 20 10gbps servers,
Uploading and downloading.
And every 3-5 hours i get hit by a wave of "googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded"
Even tho i know im no way near any API limits.
unfortunatly this looks to be a new way of google stopping people getting around the standard limits
Ive had it since august 1st
@wytv Can you test whether there is a possibility of sharing the 750g/day upload quota for SA accounts generated by a single Google Cloud project?
Im experiencing this issue as well.
Its not a gclone issue looks like something that Google has changed,
I have over 51k service accounts in rotation across over 20 10gbps servers,
Uploading and downloading.
And every 3-5 hours i get hit by a wave of "googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded"
Even tho i know im no way near any API limits.
unfortunatly this looks to be a new way of google stopping people getting around the standard limits
Ive had it since august 1st
@wytv I'm using 2 projects with 100 accounts each, and I've tested server side copy data for about 3TiB in 10 parallel transfer without any problem.
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Transferred: 5254 / 5254, 100%
Server Side Copies: 5254 @ 2.766 TiB
Elapsed time: 33m56.2s
As your server scale is large enough, imo it's better to seperate sa accounts or projects with server range, eg. 5 or 10 servers using 5 or 10 projects. It's hard to mantaince but better then all in one, or maybe decreate much more chance to consumed any hidden quotes for single sa.
@dogbutcat Have you experienced any new changes in the upload quota or do you still have no issues with your service accounts? I am getting a lot more 403 userRateLimitExceeded errors.
Can you also explain what this quota is?
"Google file owner quota (~2TB/day)"
I tried googling this but can't find any information and I'm not sure if this quota is causing an issue.
"Google file owner quota (~2TB/day)"
From all the files uploaded to google drive with the same account, it is only possible to download 2TB in one day. 2TB for all that files combined. No matter what account downloads and what file is it while it is one of the uploaded by the same account.
This limit doesn't apply if the account downloading is the same that uploaded the files
@rjenk22 Sorry for late reply. My answer is No, my service working fine for months even I didn't notice rclone has release version to v1.66.0. Maybe my daily usage haven't meet the quote? What's your using purposal and could you please giving more details? I'm using as multi-media server and bittorrent online storage pool