IBM/unitxt

consolidate the 2 "data_classification_policy" fields in the recipe

Opened this issue · 0 comments

currently we pass data_classification_policy in 2 places in the recipe

As a field and in the task_data. We need to think if and how to consolidate them.

for example:

        target = {
            "metrics": ["metrics.accuracy"],
            "source": "<<SYS>>\nYou are a helpful, respectful and honest assistant....,
            "target": " C",
            "references": [" C"],
            "task_data": '{"topic": "testing",'
            ' "question": "what?",'
            ' "choices": ["yes", "not", "maybe"],'
            ' "answer": "maybe",'
            ' "options": [" A", " B", " C"],'
            ' "metadata": {"data_classification_policy": [], "template": "templates.qa.multiple_choice.with_topic.lm_eval_harness"}'
            "}",
            "group": "unitxt",
            "postprocessors": ["processors.first_character"],
            "data_classification_policy": [],
        }

@elronbandel please advise