Task type for head to tail association
puraminy opened this issue · 9 comments
I have some data like this:
These are heads and I saved then in train.input_text
PersonX uses PersonX's ___ to obtain
PersonX uses PersonX's ___ to obtain
PersonX uses PersonX's ___ to obtain
PersonX uses PersonX's ___ to obtain
PersonX changes men 's ___
PersonX changes men 's ___
And tails are the other node in a relation. For example for "intention" relation, these could be tails, which specify what is the intention of a person of doing an action (heads above) and they are saved as train.target_text:
to have an advantage
to fulfill a desire
to get out of trouble
to be powerful
to be influential
to reform men with a bad attitude.
good about themselves
Each row corresponds to a row in tails. I tried to model them using t5, but I don't know which task_type to use. I tried summarization and the generated output is like this:
personx's ____
personx's ____
personx's ____
personx's ____
personx's ____
personx changes men's _____
personx changes men's _____
Which is obviously far from targets. It seems it jut tried to summarize them. but it must find a mapping between them.
Which task type is suitable for this common task?
Sorry, I am a little confused about the task definition.
Is the target text the prediction of ____ ?
No!, ____ are irrelevant here.They are optional and they are just to make the sentence more general.
A simile example is like below:
input: PersonX goes to store
output (intention): to buy something
Given "PersonX goes to store", it must predict "to buy something", which is the intention of going to store...
OK, I understand.
I think you can choose any task type and modify the t5_task_text to ''.
However, I don't think T5 is very suitable for this task, and GPT-2 may be more suitable.
You can use --model=GPT2Seq to use GPT-2 for Seq2Seq task.
Thanks, the result was better but not still good for T5 compared to using T5 directly from huggingface or the other github project of T5.
As for GPTSeq, it says there is no such a model:
File "/home/pouramini/TextBox/textbox/utils/utils.py", line 67, in get_model
raise NotImplementedError("{} can't be found".format(model_file_name))
NotImplementedError: gpt2seq can't be found
OK, thanks for your issue, we will check this problem.
As for GPT-2, please pull the latest version.
Could you please provide the link or code to use T5 directly from huggingface or the other github project?
We would like to check if the problem is solved.
mapping seq2seq for t5 is a very common project, and it seems the result of your project isn't good! my own code is a bit complicated to run probably, however I will provide it too... But for example, I copied part of my code from this repository and you may want to check if you can reproduce their results, particularly for generation tasks or using knowelege graphs
https://github.com/allenai/rainbow
If you find it difficult, I will try to provide my own data and code too.
Also this is a fork by mine with data provided:
https://github.com/puraminy/rainbow
You can clone it, then cd bin and run the following code. The data is proivded as atomic_train.tsv
python fine-tune.py input_text --model-dir=test --do_train
test is output directory here, however it can have a checkpoint!
However, it loads t5 model from disk and you can provide it or change the code to load it from somewhere. It uses the original t5 checkpoints and not huggingface.
OK, thanks for your help. We will try to fix this bug.