Keyword Historical Metric - RESOURCE_EXHAUSTED
NeerajG03 opened this issue · 3 comments
NeerajG03 commented
Describe the bug:
I am trying to use the KeywordPlanIdeaService and am running into the issue of RESOURCE_EXHAUSTED. This doesn't seem to be an issue always i run into this issue and there is no source for this error I have seen. The error seems to be random and not always pop up. It say retry after 30 seconds which i retiried even after one whole day and it throws this error. I retry after like 2 seconds and it works. I don't totally grasp why this might be happening and require some assistance.
Steps to Reproduce:
def generate_historical_metrics(client, customer_id,keywords):
googleads_service = client.get_service("GoogleAdsService")
keyword_plan_idea_service = client.get_service("KeywordPlanIdeaService")
request = client.get_type("GenerateKeywordHistoricalMetricsRequest")
request.customer_id = customer_id
request.keywords.extend(keywords)
request.geo_target_constants.append(
googleads_service.geo_target_constant_path("2840")
)
request.keyword_plan_network = (
client.enums.KeywordPlanNetworkEnum.GOOGLE_SEARCH
)
request.language = googleads_service.language_constant_path("1000")
response = keyword_plan_idea_service.generate_keyword_historical_metrics(
request=request
)
output = []
for result in response.results:
metrics = result.keyword_metrics
newval = []
for each in list(metrics.monthly_search_volumes):
newval.append({"year":each.year,"month":each.month,"searches":each.monthly_searches})
output.append({
"keyword":result.text,
"variants":str(result.close_variants) if result.close_variants else 'None',
"searches":metrics.avg_monthly_searches,
"competition_level":"Low" if metrics.competition == 2 else "Medium" if metrics.competition==3 else "High" if metrics.competition==4 else "Level Unknown",
"competition_index" : metrics.competition_index,
"lowest_bid": metrics.low_top_of_page_bid_micros,
"highest_bid": metrics.high_top_of_page_bid_micros,
"history":newval,
"growing":is_trajectory_upward(newval)
})
print(output[-1])
I run the above code like so:
googleads_client = GoogleAdsClient.load_from_storage(path="GoogleAPI/google-ads.yaml",version="v15")
customer_id = 'xxxxxxxxx'
keywords = ["word1","word2,"word3"]
generate_historical_metrics(googleads_client, customer_id,keywords)
Expected behavior:
Print the output one after the other.
which it does sometimes:
When it works:
Ouput:
{'keyword': 'word1', 'variants': 'None', 'searches': 210, 'competition_level': 'Low', 'competition_index': 4, 'lowest_bid': 0, 'highest_bid': 0, 'history': [{'year': 2023, 'month': 3, 'searches': 110}, {'year': 2023, 'month': 4, 'searches': 1000}, {'year': 2023, 'month': 5, 'searches': 90}, {'year': 2023, 'month': 6, 'searches': 90}, {'year': 2023, 'month': 7, 'searches': 90}, {'year': 2023, 'month': 8, 'searches': 90}, {'year': 2023, 'month': 9, 'searches': 110}, {'year': 2023, 'month': 10, 'searches': 140}, {'year': 2023, 'month': 11, 'searches': 140}, {'year': 2023, 'month': 12, 'searches': 90}, {'year': 2023, 'month': 13, 'searches': 110}, {'year': 2024, 'month': 2, 'searches': 110}], 'growing': False}
{'keyword': 'word2', 'variants': 'None', 'searches': 4400, 'competition_level': 'Low', 'competition_index': 2, 'lowest_bid': 29970407, 'highest_bid': 581105000, 'history': [{'year': 2023, 'month': 3, 'searches': 4400}, {'year': 2023, 'month': 4, 'searches': 5400}, {'year': 2023, 'month': 5, 'searches': 3600}, {'year': 2023, 'month': 6, 'searches': 3600}, {'year': 2023, 'month': 7, 'searches': 3600}, {'year': 2023, 'month': 8, 'searches': 3600}, {'year': 2023, 'month': 9, 'searches': 3600}, {'year': 2023, 'month': 10, 'searches': 4400}, {'year': 2023, 'month': 11, 'searches': 4400}, {'year': 2023, 'month': 12, 'searches': 4400}, {'year': 2023, 'month': 13, 'searches': 5400}, {'year': 2024, 'month': 2, 'searches': 5400}], 'growing': True}
When it throws error:
Output:
Request made: ClientCustomerId: 5959491719, Host: googleads.googleapis.com, Method: /google.ads.googleads.v15.services.KeywordPlanIdeaService/GenerateKeywordHistoricalMetrics, RequestId: XjjJbCCSYYPEz2vJ3bWI_A, IsFault: True, FaultMessage: Resource has been exhausted (e.g. check quota).
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 79, in error_remapped_callable
return callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 277, in __call__
response, ignored_call = self._with_call(
^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 332, in _with_call
return call.result(), call
^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 437, in result
raise self
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 315, in continuation
response, call = self._thunk(new_method).with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 343, in with_call
return self._with_call(
^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 332, in _with_call
return call.result(), call
^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 437, in result
raise self
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 315, in continuation
response, call = self._thunk(new_method).with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 343, in with_call
return self._with_call(
^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 329, in _with_call
call = self._interceptor.intercept_unary_unary(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/ads/googleads/interceptors/exception_interceptor.py", line 99, in intercept_unary_unary
self._handle_grpc_failure(response)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/ads/googleads/interceptors/exception_interceptor.py", line 71, in _handle_grpc_failure
raise self._get_error_from_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 315, in continuation
response, call = self._thunk(new_method).with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1177, in with_call
return _end_unary_response_blocking(state, call, True, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1003, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.RESOURCE_EXHAUSTED
details = "Resource has been exhausted (e.g. check quota)."
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2404:6800:4007:818::200a%5D:443 {grpc_message:"Resource has been exhausted (e.g. check quota).", grpc_status:8, created_time:"2024-02-17T20:06:36.312932+05:30"}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/ng/Documents/Github/python-api/GoogleAPI/local-test-file2.py", line 74, in <module>
generate_historical_metrics(googleads_client, customer_id,keywords)
File "/Users/ng/Documents/Github/python-api/GoogleAPI/local-test-file2.py", line 36, in generate_historical_metrics
response = keyword_plan_idea_service.generate_keyword_historical_metrics(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/ads/googleads/v15/services/services/keyword_plan_idea_service/client.py", line 539, in generate_keyword_historical_metrics
response = rpc(
^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 81, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.ResourceExhausted: 429 Resource has been exhausted (e.g. check quota). [type_url: "type.googleapis.com/google.ads.googleads.v15.errors.GoogleAdsFailure"
value: "\n-\n\002X\002\022\'Too many requests. Retry in 30 seconds.\022\026XjjJbCCSYYPEz2vJ3bWI_A"]
Client library version and API version:
Client library version:
Google Ads API version:
Request/Response Logs:
[2024-02-17 20:10:43,678 - INFO] Request
-------
Method: /google.ads.googleads.v15.services.KeywordPlanIdeaService/GenerateKeywordHistoricalMetrics
Host: googleads.googleapis.com
Headers: {
"developer-token": "REDACTED",
"login-customer-id": "xxxxxxxxx",
"x-goog-api-client": "gl-python/3.11.7 grpc/1.60.1 gax/2.16.2 gccl/23.0.0 pb/4.25.2",
"x-goog-request-params": "customer_id=xxxxxxxx"
}
Request: customer_id: "xxxxxxxxx"
keywords: "ai analytics"
keywords: "marketing"
language: "languageConstants/1000"
geo_target_constants: "geoTargetConstants/2840"
keyword_plan_network: GOOGLE_SEARCH
Response
-------
Headers: {
"google.ads.googleads.v15.errors.googleadsfailure-bin": "\n-\n\u0002X\u0002\u0012'Too many requests. Retry in 30 seconds.\u0012\u0016N5ryRsfeFjxtVKmARHpwCQ",
"grpc-status-details-bin": "\b\b\u0012/Resource has been exhausted (e.g. check quota).\u001a\u0001\nDtype.googleapis.com/google.ads.googleads.v15.errors.GoogleAdsFailure\u0012G\n-\n\u0002X\u0002\u0012'Too many requests. Retry in 30 seconds.\u0012\u0016N5ryRsfeFjxtVKmARHpwCQ",
"request-id": "N5ryRsfeFjxtVKmARHpwCQ"
}
Fault: {}
[2024-02-17 20:10:43,679 - WARNING] Request made: ClientCustomerId: xxxxxxxxx, Host: googleads.googleapis.com, Method: /google.ads.googleads.v15.services.KeywordPlanIdeaService/GenerateKeywordHistoricalMetrics, RequestId: N5ryRsfeFjxtVKmARHpwCQ, IsFault: True, FaultMessage: Resource has been exhausted (e.g. check quota).
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 79, in error_remapped_callable
return callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 277, in __call__
response, ignored_call = self._with_call(
^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 332, in _with_call
return call.result(), call
^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 437, in result
raise self
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 315, in continuation
response, call = self._thunk(new_method).with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 343, in with_call
return self._with_call(
^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 332, in _with_call
return call.result(), call
^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 437, in result
raise self
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 315, in continuation
response, call = self._thunk(new_method).with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 343, in with_call
return self._with_call(
^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 329, in _with_call
call = self._interceptor.intercept_unary_unary(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/ads/googleads/interceptors/exception_interceptor.py", line 99, in intercept_unary_unary
self._handle_grpc_failure(response)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/ads/googleads/interceptors/exception_interceptor.py", line 71, in _handle_grpc_failure
raise self._get_error_from_response(response)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_interceptor.py", line 315, in continuation
response, call = self._thunk(new_method).with_call(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1177, in with_call
return _end_unary_response_blocking(state, call, True, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/_channel.py", line 1003, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.RESOURCE_EXHAUSTED
details = "Resource has been exhausted (e.g. check quota)."
debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B2404:6800:4007:816::200a%5D:443 {created_time:"2024-02-17T20:10:43.674219+05:30", grpc_status:8, grpc_message:"Resource has been exhausted (e.g. check quota)."}"
>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/neerajgopalakrishnan/Documents/Github/python-api/GoogleAPI/local-test-file2.py", line 77, in <module>
generate_historical_metrics(googleads_client, customer_id,keywords)
File "/Users/neerajgopalakrishnan/Documents/Github/python-api/GoogleAPI/local-test-file2.py", line 39, in generate_historical_metrics
response = keyword_plan_idea_service.generate_keyword_historical_metrics(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/ads/googleads/v15/services/services/keyword_plan_idea_service/client.py", line 539, in generate_keyword_historical_metrics
response = rpc(
^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
return wrapped_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 81, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.ResourceExhausted: 429 Resource has been exhausted (e.g. check quota). [type_url: "type.googleapis.com/google.ads.googleads.v15.errors.GoogleAdsFailure"
value: "\n-\n\002X\002\022\'Too many requests. Retry in 30 seconds.\022\026N5ryRsfeFjxtVKmARHpwCQ"
]
Anything else we should know about your project / environment:
I don't think it is related to environment as it works sometimes. I'm just knowledgable about why the quota is being crossed
NeerajG03 commented
This might be a stupid issue so feel free to ask me questions and i an ready to answer. I just am unable to figure out for the life of me why this is happening.
MaxBarbet commented
You are limited to approximately 1 request per second, and the scripts often checks if the quota is passed each 5 or 30 seconds.
Using a time.sleep(1) between each request will fix the issue.
Furthermore, you have a 15k request a day limit on the basic access and it is calculed based on the last 24 hours.
Hope this helps,
BenRKarl commented
@NeerajG03 let me know if the suggestions from @MaxBarbet help. If not, I recommend following the support steps on this page to resolve the issue, as it doesn't look to be an issue with this library.