Some questions about attention model!

[Imorton-zd]

Hi, @codekansas, thanks for sharing the project. I have read the main file about attention model to question answering and the paper you suggested. However, I have some questions as following:
Take insuranceqa.py for example.
1.Is the input of your data the word frequence index? Would you give me an actual example of your input data?
q_data, ag_data, ab_data, targets = get_data(data_sets[0])
2. What is the "mrr" in your code? A evaluation measure?
def get_mrr(model, questions, all_answers, n_good, n_eval=-1):
3. Why are the targets all zeros?

def get_data(f_name):
....
        targets += [0] * len(bad_answers)

4 I find you load the weights of test model, as following:
test_model.load_weights(os.path.join(models_path, 'iqa_model_for_training_iter_900.h5'))
Where are the weights of test model? The weights of trained model can transfer to test model?

5 Similar to my task (figuring out how similar two tweets), I only need to get the similarity of each tweet pair. But I have to set a threshold value, which is used to determine whether two tweets are from the same person or not (If the similarity greater than threshold value, the pair is from the same person, vice versa). Therefore, do you think what the threshold value should be set? Or other method to leverage similarity to determine whether two tweets are from the same person or not?

[codekansas]

I should warn you that a lot of the code is pretty messy since I've just been trying different things out.

1. The input is a sentence, encoded as indices. So a sentence "the cat in the hat" might be encoded as [1, 2, 3, 1, 4]. This is done by constructing a dictionary of words and taking the index of the word in the dictionary (with a dummy index if the word doesn't exist).
2. MRR means Mean Reciprocal Rank, it is a common metric in question answering. If you have a selection of good answers out of all your answers, the reciprocal rank is 1 over the highest rank of a good answer. Then you take the mean of all of those to get MRR.
3. In my model, I was minimizing hinge loss between a good and a bad answer, using something like merge([good, bad], mode=lambda x: 0.2 - x[0] + x[1]). So you want this to go to 0, so that good (the cosine similarity between the question encoding and the good answer encoding) is maximized and bad is minimized, up to a margin. This is done in the two papers.
4. That was just an artifact of messing around. I didn't want to retrain a model to evaluate it, but some stuff is out of order because I commented different things.
5. I'd say you could figure that out from your training set, like run a bunch of training sets through and determine how similar tweets from the same person usually are. I'd imagine there would be a lot of noise in that kind of task though. I'd be interested to hear if it actually works.

I don't think this is really Keras related though, so I'd suggest closing this issue and messaging me directly if you have more questions. My email is on my Github page. Also, I started writing some of this stuff up here but it isn't complete yet.