Quota exceeded: Your table exceeded quota for imports or query appends per table.
mattcobb opened this issue · 5 comments
Setup a logpush to this gcp cloud function. Seemed to be working fine, but then we couldn't find some RayIDs we saw in our logs. On further investigation, we found the google function was logging the quota exceeded error below. The table insert quota is 1500/table/day. CF is creating a log every 30secs which is 2880/day so the quota gets exceeded. Is there a way to create a file no more than once per min (1440/day). Separately we are looking into doubling the Google quota.
Default
2021-05-06 15:07:52.007 PDTgcsbq2ddmbm47hmmcn Starting job for logs/20210506/20210506T220721Z_20210506T220751Z_477368e1.log.gz
Default
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn { Error: Quota exceeded: Your table exceeded quota for imports or query appends per table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors
Default
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn id:
Default
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn at request (/workspace/node_modules/@google-cloud/bigquery/build/src/index.js:888:23)
Default
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn 'pryon-demo-dev-215016:US.e6ef4b65-8472-4ab8-8142-6b5b4e568b1d',
Default
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn at Util.handleResp (/workspace/node_modules/@google-cloud/common/build/src/util.js:141:9)
Default
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn selfLink:
Default
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn 'https://www.googleapis.com/bigquery/v2/projects/pryon-demo-dev-215016/jobs/e6ef4b65-8472-4ab8-8142-6b5b4e568b1d?location=US'
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn at retryRequest (/workspace/node_modules/@google-cloud/common/build/src/util.js:422:22)
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn at onResponse (/workspace/node_modules/retry-request/index.js:200:7)
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn user_email: 'pryon-demo-dev-215016@appspot.gserviceaccount.com'
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn configuration: { load: [Object], jobType: 'LOAD' }
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn at /workspace/node_modules/teeny-request/build/src/index.js:222:13
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn jobReference:
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn at process._tickCallback (internal/process/next_tick.js:68:7)
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn code: undefined,
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn { projectId: 'pryon-demo-dev-215016'
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn jobId: 'e6ef4b65-8472-4ab8-8142-6b5b4e568b1d'
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn errors:
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn [ { reason: 'quotaExceeded'
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn location: 'US' }
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn location: 'load_job_per_table.long',
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn statistics:
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn message:
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn { creationTime: '1620338872353'
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn startTime: '1620338872502'
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn 'Quota exceeded: Your table exceeded quota for imports or query appends per table. For more information, see https://cloud.google.com/bigquery/troubleshooting-errors' } ],
2021-05-06 15:07:52.591 PDTgcsbq2ddmbm47hmmcn endTime: '1620338872502' },
Thank you!
@shagamemnon Any update on #70 release date?
Hey there! I am having the same issue and I was wondering if there were any updates. Thanks!
It has been a month, any update on this? This feature is currently not viable as the quota kicks in around half way thru the day.