anthropics/anthropic-sdk-python

Wrong request options in retries

hahampis opened this issue · 4 comments

hahampis commented

We are using AsyncAnthropicBedrock class. We noticed that when a request is automatically retried, the request options do not have the correct values, resulting in 400 - Bad request. Example logs (actual content redacted):

2024-07-12 09:56:25,645 - anthropic._base_client - DEBUG - Request options: {'method': 'post', 'url': '/model/anthropic.claude-3-haiku-20240307-v1:0/invoke', 'timeout': 600, 'files': None, 'json_data': {'max_tokens': 4096, 'messages': [{'role': 'user', 'content': '...', , 'temperature': 0.0, 'top_k': 1, 'anthropic_version': 'bedrock-2023-05-31'}}
2024-07-12 09:56:25,658 - anthropic._base_client - DEBUG - Encountered Exception	
2024-07-12 12:56:25.665	Traceback (most recent call last):
...
2024-07-12 09:56:25,665 - anthropic._base_client - DEBUG - 1 retry left
2024-07-12 09:56:25,665 - anthropic._base_client - INFO - Retrying request to /v1/messages in 0.924755 seconds
2024-07-12 09:56:26,591 - anthropic._base_client - DEBUG - Request options: {'method': 'post', 'url': '/model/None/invoke', 'timeout': 600, 'files': None, 'json_data': {'max_tokens': 4096, 'messages': [{'role': 'user', 'content': '...', 'temperature': 0.0, 'top_k': 1, 'anthropic_version': 'bedrock-2023-05-31'}}
2024-07-12 09:56:26,773 - httpx - INFO - HTTP Request: POST https://bedrock-runtime.eu-central-1.amazonaws.com/model/None/invoke "HTTP/1.1 400 Bad Request"
2024-07-12 09:56:26,775 - anthropic._base_client - DEBUG - HTTP Request: POST https://bedrock-runtime.eu-central-1.amazonaws.com/model/None/invoke "400 Bad Request"
2024-07-12 09:56:26,775 - anthropic._base_client - DEBUG - Encountered httpx.HTTPStatusError

This started happening with version 0.31.0. Doesn't happen with version 0.30.1.
Maybe has something to do with #580 ?

RobertCraigie commented

Thanks for the report, I can reproduce this, working on a fix.

  • 👍1
RobertCraigie commented

this will be fixed in the next release: #585, v0.31.1.

yasyf commented

I think this is happening with vertex too.. upon retry you get KeyError: 'model', which I think happens because the options payload is mutated. Fix here: #599

RobertCraigie commented

Sorry about that @yasyf, that will be fixed in the next release: #595

Until it's released (hopefully later today) you can either downgrade or install from github, e.g. python -m pip install -U git+https://github.com/anthropics/anthropic-sdk-python@next.

  • ❤️1