Feature request: Request rate limitation

[rabbull]

Hi zurawiki,

My OpenAI account is currently kept unpaid, which resulted in rate limitation failures on large (maybe?) patchsets:

$ git commit
🤖 Asking GPT-3 to summarize diffs...
2023-02-09T16:33:58.976Z WARN  [gptcommit::openai] skipping... token count: 4552 < 4097
2023-02-09T16:33:58.980Z WARN  [gptcommit::openai] skipping... token count: 4092 < 4097
2023-02-09T16:33:58.988Z WARN  [gptcommit::openai] skipping... token count: 4310 < 4097
2023-02-09T16:33:58.988Z WARN  [gptcommit::openai] skipping... token count: 4252 < 4097
2023-02-09T16:33:58.992Z WARN  [gptcommit::openai] skipping... token count: 17654 < 4097
Error: Unexpected JSON response:
{"error":{"code":null,"message":"Rate limit reached for default-text-davinci-003 in organization org-xxxxxx on requests per min. Limit: 60.000000 / min. Current: 70.000000 / min. Contact support@openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.","param":null,"type":"requests"}}

$ git diff --staged --shortstat
 28 files changed, 2854 insertions(+)

The error occurs mid-execution, so the requests before the error were dropped. Retries may cause additional overhead in the costing. (related to #41)

Switching to a mature OpenAI API client (#20) could probably solve this problem, but before doing so, an additional configuration to limit the request rate will also come in handy.

[zurawiki]

Acknowledged. I think it would make sense to make this configurable with a setting like:

openai.max_retries and set it 0 by default. Any feedback on that?

[rabbull]

Acknowledged. I think it would make sense to make this configurable with a setting like:

openai.max_retries and set it 0 by default. Any feedback on that?

Look good to me.