BaranziniLab/KG_RAG

run time

DayanaYuan opened this issue · 7 comments

DayanaYuan commented

It's been running all night with no results
def fetch_GPT_response(instruction, system_prompt, chat_model_id, chat_deployment_id, temperature=0):
print('Calling OpenAI...')
print("1.4\n")
response = openai.ChatCompletion.create(
temperature=temperature,
deployment_id=chat_deployment_id,
model=chat_model_id,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": instruction}
]
)
print("1.5\n")
if 'choices' in response
and isinstance(response['choices'], list)
and len(response) >= 0
and 'message' in response['choices'][0]
and 'content' in response['choices'][0]['message']:
return response['choices'][0]['message']['content']
else:
return 'Unexpected response'

after print("1.4\n"), the operation is stuck. No result

karthiksoman commented

Hi @yangyangyang-github
This is a typical openai API call.
If the function is going in an endless loop, can you please double check your openai credentials that was provided in the config.yaml file?
Also, we have already put guardrails in the api call function to stop making calls after some sufficient time using retry module. Please refer here. Hence, if you are using the same functionality, it should not go to an endless loop.
Let me know how things turn out for you.

DayanaYuan commented

Hi @yangyangyang-github This is a typical openai API call. If the function is going in an endless loop, can you please double check your openai credentials that was provided in the config.yaml file? Also, we have already put guardrails in the api call function to stop making calls after some sufficient time using retry module. Please refer here. Hence, if you are using the same functionality, it should not go to an endless loop. Let me know how things turn out for you.

I have added openai.api_key parameters to the file. What exactly is the openai credentials that was provided in the config.yaml file? May I have a look, please?

karthiksoman commented

You should have a file named '.gpt_config.env' and store it in your $HOME path. Content of the file should be in the following format:

API_KEY='openai api key'
API_VERSION='this is optional'
RESOURCE_ENDPOINT='this is optional'

DayanaYuan commented
karthiksoman commented

The file contains API credentials, which, like any other sensitive information, should ideally not be shared publicly. Hope you get it :)
Feel free to reach out if you need further assistance!

DayanaYuan commented
karthiksoman commented

@yangyangyang-github
Did you check if this is a memory issue? We are not using quantized versions of llama here, hence it could take a good chunk of memory. If you see here, you can see the size of the tensors for llama-13b and compare it with the memory of the machine that you are using.

I tried using llama-13b on p3.8x.large AWS instance which has following specs:
4 Tesla V100 GPU
64 GB GPU memory
32 vCPU
244 GB RAM