kjappelbaum/gptchem

empty prediction

Opened this issue · 3 comments

venuv commented

@kjappelbaum in various runs of the example you provided, I get a prediction of either None, or in this case a silent return. any tips on debugging. my openai_api_key seems to not be an issue - TIA, Venu
Screenshot 2023-04-15 at 5 30 37 PM

I created this method which fixed my use case in fine tuning on binary 'Active' or 'Inactive' labels:

# Import the original class from the external module
from gptchem.extractor import BaseExtractor

class ClassificationExtractorStrings(BaseExtractor):
    """Extract strings from completions of classification tasks."""
    def extract(self, data, **kwargs) -> str:
        return self.split(data).strip()
U0M0Z commented

@kjappelbaum I've encountered the same issue. I've tried running both the Classification and Regression tasks in the "High-level wrapper" section and got empty predictions in both scenarios. There shouldn't be any issues with the OpenAI key since I've been using it already for other tasks. I can also track the requests to the API in the usage section of OpenAI platform.

Regression task:

gpt-chem-error-1

Classification task

gpt-chem-error-2

Thanks for all the reports; I have been busy with my Ph.D. defense but have been working on things to make all of this more robust, and hope to push them soon!