bananaml/serverless-template

EleutherAI-gpt-neo-2.7B taking about 2 minutes to respond

Opened this issue · 0 comments

CodeLikeThe90s commented

EleutherAI-gpt-neo-2.7B taking about 2 minutes to respond for prompt with max_length under 100
Shouldn't respond time be faster when running on GPU?